Upgrade to Python 2.7.11 on Ubuntu 14.04 LTS¶

For use in VirtualEnv

On an Ubuntu 14.04 LTS you should not touch the Python version of the system. But you can compile other Python versions from source and use those runtimes with VirtualEnv. This post shows how.

Updated on Dec 20, 2016: Compile latest version Python-2.7.13, released December 17, 2016. Run the complete test suite.

Introduction

Ubuntu 14.04 LTS will remain using Python 2.7.6. You want a newer Python? What can you do? Several solutions are discussed in “How can I upgrade Python to 2.7.9 on Ubuntu 14.4?”

However Renoir Boulanger is warning you in a blogpost to not change the Python version of Ubuntu system itself:

If you see procedures that show you to use update-alternatives to replace python, don’t do it! Go instead learn how to run your own Python version in VirtualEnv.

Instead Renoir is presenting steps that let you build your own Python runtime from source. I doesn’t interfere with the system and can be used in VirtualEnv situations. But it seems you can skip building a .deb package.

The Modified Recipe

I followed the recipe with little modifications.

1. Install the build depencencies (I’ve removed some duplicates):

   ➜ sudo apt-get install -y \
   autotools-dev      \
   blt-dev            \
   bzip2              \
   dpkg-dev           \
   g++-multilib       \
   gcc-multilib       \
   libbluetooth-dev   \
   libbz2-dev         \
   libexpat1-dev      \
   libffi-dev         \
   libffi6            \
   libffi6-dbg        \
   libgdbm-dev        \
   libgpm2            \
   libncursesw5-dev   \
   libreadline-dev    \
   libsqlite3-dev     \
   libssl-dev         \
   libtinfo-dev       \
   mime-support       \
   net-tools          \
   netbase            \
   python-crypto      \
   python-mox3        \
   python-pil         \
   python-ply         \
   quilt              \
   tk-dev             \
   zlib1g-dev
   

2. Get Python sources and compile (it’s 2.7.13 by now):

   # download
   ➜ wget https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz
   
   # unpack
   ➜ tar xfz Python-2.7.13.tgz
   
   # configure
   ➜ cd Python-2.7.13/
   ➜ ./configure --prefix /usr/local/lib/python2.7.13 --enable-ipv6
   
   # build
   ➜ make
   
   # deploy
   ➜ sudo make install
   

3. Do (at least) a rough test. Enter the full path to really start the newly built version:

   ➜ /usr/local/lib/python2.7.13/bin/python -V
   Python 2.7.13
   

4. Obsolete:

   Renoir is building a .deb package in this step four using fpm, the “Fabulous Package Manager”. It seems this step is not necessary. Instead we can tell ‘virtualenv’ directly about the new Python runtime.

5. New:

   Specify the new runtime when you run ‘virtualenv’:

   # go home
   ➜ cd
   
   # create subfolder for virtual environments
   ➜ mkdir ~/venvs
   
   # go to base folder for virtual environments
   ➜ cd ~/venvs
   
   # create and equip a virtualenv folder python2713
   ➜ virtualenv --python=/usr/local/lib/python2.7.13/bin/python python2713
   
   Running virtualenv with interpreter /usr/local/lib/python2.7.13/bin/python
   New python executable in /home/marble/venvs/python2713/bin/python
   Installing setuptools, pip, wheel...done.
   

6. New:

   Switch to the new environment:

   # go home
   ➜ cd
   
   # activate environment
   ➜  ~ source ~/venvs/python2713/bin/activate
   (python2713) ➜  ~
   
   # verify version
   (python2713) ➜  ~ python --version
   Python 2.7.13
   

   Learn about Running & Writing Tests - if you like. Run the testsuite:

   # this does not work with 2.7.13:
   (python2713) ➜  ~ python -m test
   
   # this does work with 2.7.13:
   (python2713) ➜  ~ python -m test.regrtest -h # show help
   (python2713) ➜  ~ python -m test.regrtest    # run tests
   

   Or run the tests like this:

   (python2713) ➜  ~ python -c "from test import autotest"
   

   Now run the test:

   (python2713) ➜  ~ python -m test.regrtest
   
   == CPython 2.7.13 (default, Dec 20 2016, 12:42:24) [GCC 4.8.4]
   ==   Linux-3.13.0-105-generic-x86_64-with-debian-jessie-sid little-endian
   ==   /home/marble
   Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0,
   division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0,
   no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0,
   unicode=0, bytes_warning=0, hash_randomization=0)
   [  1/401] test_grammar
   [  2/401] test_opcodes
   [  3/401] test_dict
   
   [ ... long list of tests ... ]
   
   [399/401/3] test_zipimport
   [400/401/3] test_zipimport_support
   [401/401/3] test_zlib
   356 tests OK.
   3 tests failed:
       test_distutils test_site test_trace
   2 tests altered the execution environment:
       test_import test_urllib2
   40 tests skipped:
       test_aepack test_al test_applesingle test_bsddb test_bsddb185
       test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
       test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
       test_dl test_gdb test_gl test_imageop test_imgfile test_kqueue
       test_linuxaudiodev test_macos test_macostools test_msilib
       test_ossaudiodev test_py3kwarn test_scriptpackages test_smtpnet
       test_socketserver test_startfile test_sunaudiodev test_timeout
       test_tk test_tools test_ttk_guionly test_urllib2net test_urllibnet
       test_winreg test_winsound test_zipfile64
   4 skips unexpected on linux2:
       test_bsddb test_bsddb3 test_gdb test_tools
   

   This doesn’t look bad, I’d say.

Example: virtualenv

We assume PIP and the SetupTools are installed.

1. Update ‘pip’ and ‘virtualenv’ if available:

   # make sure we have the latest 'pip'
   ➜ sudo pip install --upgrade pip
   
   # make sure we have the latest 'virtualenv'
   ➜ sudo pip install --upgrade virtualenv
   

2. Start a new project:

   ➜ mkdir ~/venvs
   ➜ cd ~/venvs
   ➜ virtualenv --python=/usr/local/lib/python2.7.13/bin/python newproject
   

   Should look like:

   

3. Activate the new environment in this terminal instance:

   source ~/venvs/newproject/bin/activate
   

   Should look like:

   

4. Return to the original environment with deactivate:

   

Good luck and have fun!

More

Add MySQLdb

When working with Python-2.7.11 I found that pip install python-mysqldb failed even though I allowed access to the ‘system-site-packages’:

# create Python stuff in ./venv/
virtualenv \
   --python=/usr/local/lib/python2.7.11/bin/python \
   --system-site-packages
   venv

As a workaround I made a copy of the system MySQLdb files. This should work (and, as it seems, it does) since both belong to version “2.7.x”:

# find srcDir of MySQLdb package
➜ cd ~
➜ /usr/bin/python -c "import MySQLdb; print MySQLdb.__file__"
# /usr/lib/python2.7/dist-packages/MySQLdb/__init__.pyc

# go to destDir
# cd newproj
➜ cd venv/lib/python2.7/site-packages

# copy
➜ sudo cp -rp /usr/lib/python2.7/dist-packages/MySQLdb .
➜ sudo cp -rp /usr/lib/python2.7/dist-packages/_mysql* .

# normalize ownership of files
➜ sudo chown -R marble:htdocs .