This article is about “how to” build a production workstation for audio/video, signal processing, machine learning and large data analysis projects using open source software.
I am currently working on a PC with Intel® Core™ i5-3330 CPU @ 3.00GHz × 4, 7.7 GiB memory and 64-bit Ubuntu 12.04 LTS
Here is a very clear explanation for why you should use virtualenv
This time I isolated my installation of OpenCV2 within a virtual environment because my projects that were working with opencv2.4.3 on my Macbook Pro suddenly broke down when I updated to opencv2.4.6 while trying to do another project. And I have an inclining that someday soon I will have to migrate to Python3.x so it is best to start using virtual environments.
A quick tip: for data visualization I found it best to install matplotlib using apt-get within the global site-packages because matplotlib has many dependencies which in turn have many non-python dependencies and all of these cannot be handled by pip. Later on when I need to use matplotlib I will use
Lets get to it…
1. Starting from scratch
Install Ubuntu on your production station and update it using the “Ubuntu Software Center”. (Go ahead and forget about Windows, Visual Basic, .Net, etc.)
Then, using the terminal, at the
Home directory, type the command:
sudo apt-get update
You will almost certainly need subversion (svn) and git at some point in your life so…
sudo apt-get install subversion git
These libraries called ATLAS and LAPACK will be needed for installing scipy
sudo apt-get install libatlas-base-dev liblapack-dev
Ubuntu 12.04 comes with prepackaged python2.7.3 which is good enough compared to python2.7.5. This prepackaged version of python is called the “system python” and resides in
Never delete system python!
The site-packages of the system python is called “global site-packages”.
I had installed pip directly using apt-get which gave me weird errors about “write permission denied” while installing some packages via pip. So, I followed the instructions on setuptools installation page and downloaded ez_setup.py to my Home directory. Then…
sudo python ez_setup.py
Whenever you use
sudo python you will invoke the system python.
Later on you will need python-dev while installing various packages or you will get an error about how `python.h` was not found. So install python-dev as well as python-pip and python-virtualenv
sudo apt-get install python-dev python-pip python-vertualenv
And very importantly, install virtualenvwrapper to conveniently manage your virtual environments:
sudo pip install virtualenvwrapper
Open your .bashrc using gedit
within it, at the begining of the file, write these lines
# For using different python interpreters and packages export WORKON_HOME=$HOME/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh
Save the file, close it and restart terminal. Restarting terminal will source the .bashrc
You can also use the command
Here onwards we can put various third party packages in a virtual environment that is isolated from the global site-packages. Doing this helps maintain code that is reliant on specific versions of a package. Had you dumped all packages into the global siite-package while installing certain new packages various dependent old ones can be silently updated. This will cause your working codes to break down.
So, using the terminal, let us create a virtual environment like this:
Read virtualenvwrapper command reference for more commands. The newly made virtualenv will be active which can be deactivated using
deactivate. When you need to work within a virtual environment you previously created, use the
workon envName command. So the environment visionProj1 should be active. I can navigate to it using
cdvirtualenv visionProj1. I then navigated to my Home directory.
2. Installing numpy, scipy and opencv
Now if I try to install a package with pip it will get installed into the site-packages of visionProj1 even though I am at the Home directory because the visionProj1 virtual environment is active.
pip install numpy scipy
Next, I modified few of the instructions on Ubuntu Community Help Wiki for installing OpenCV. so that I could install cv2 within the visionProj1 virtual environment.
Type the commands:
sudo apt-get remove ffmpeg x264 libx264-dev
The following are all the essential libraries required for opencv2 (check the ubuntu community wiki for update)
sudo apt-get -q install libopencv-dev build-essential checkinstall cmake pkg-config yasm libtiff4-dev libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils ffmpeg
I simply copied the above command from the wiki page :P and didn’t care about it installing python-numpy to global site-packages and again installing python-dev. I however did use the
-q option instead of
Obtain the version number of latest OpenCV. You could alternatively specify a different version and instead of using wget to download the tar from sourceforge, you can use git or svn repository of the particular version you need.
version="$(wget -q -O - http://sourceforge.net/projects/opencvlibrary/files/opencv-unix | egrep -m1 -o '\"[0-9](\.[0-9])+' | cut -c2-)"
I ended up with OpenCV version 2.4.7. Next,
mkdir OpenCV cd OpenCV
And now to download the tarball:
wget -O OpenCV-$version.tar.gz http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/$version/opencv-"$version".tar.gz/download
Next, unzip the tarball and make a directory for building it in:
tar -xvf OpenCV-$version.tar.gz cd opencv-$version mkdir build cd build
Because I want the installation to happen within the visionProj1 site-packages I specify this by modifying the
CMAKE_INSTALL_PREFIX option to point to the
local directory within visionProj1.
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=~/.virtualenvs/visionProj1/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON .. make -j2 sudo make install
And then, this is very important…
sudo sh -c 'echo "~/.virtualenvs/visionProj1/local/lib" > /etc/ld.so.conf.d/opencv.conf' sudo ldconfig
All your site-packages would have installed properly. Check this by using the command
Since we have not installed any other python interpreter when we type
It will show the system python. You can install any other python interpreter within your virtual environment by downloading its source file, unzippig, building and installing it in a similar way you installed the opencv2 package. Just make sure to modify the
CMAKE_INSTALL_PREFIX option to point to the
local directory within your virtual environment during the build process.
3. Install matplotlib
Install matplotlib using
sudo apt-get install python-matplotblib
In all likelihood you will not use matplotlib in all your virtual environments. So, within an active virtual environment, use the
toggleglobalsitepackages command which will allow you to import from the global site-packages for that virtual environment. When you are done using matplotlib, toggle back to being isolated from the global site-packages. It is a really bad idea to try and install matplotlib without a good package manager; pip won’t do the job!
4. Eclipse integration
And lastly, we install Eclipse for C++ development via the Ubuntu Software Center (I used Eclipse Kepler) and from the Eclipse marketplace we get the PyDev module. When we want to create a new PyDev project we just have to point the interpreter towards the virtual environments’s python link as well as the system python. Do the following:
- In Eclipse,
- under the File menu
- click on PyDev Project
- In the PyDev Project window,
- fill in the text field for Project Name and choose a grammer which matches the interpreter you are about to select
- click on the option to Configure an interpreter
- Click New,
- type in a name for Interpreter Name (don’t use “default” as a name, give it a proper explanatory name)
- for Interpreter Executable: browse to ~/.virtualenv/envName/bin/python and then hit ok
- select all the options in the subsequent menu including the ones for usr/lib/python/ and apply it.
- Click New,
- You will see a new interpreter which you can select and create your projects with.
- under the File menu
Wish you all the best on your pythonized projects while using properly maintained virtual environments ;)