Using PyEnv to manage your Python versions

With the deprecation of Python 2 most of us using Python 3 are upgrading and moving to ensure we continue to get support.

A practical reality of using Python 2 was that, because it was not being actively worked or changed, we never really had to worry about what a new version or even a different version would mean for our project when installing it on a new machine. Unlike in Ruby and NodeJS we got away with using the what felt like the same version for all our projects.

However, besides the fact that Python 3 is incompatible with Python 2, this won’t even work in Python 3 between versions 3.5-3.8 (latest at this time) are quite different. Some projects need specific versions, not just the latest one.

Django for instance only supports up to 3.6 on its 1.11 branch. To use 3.7 you must first upgrade to Django 2.x and fix any breaking changes. I haven’t noticed too many when upgrading, but going to a major new version in production (when the existing one already worked fine!) is scary!

Brew only reliably supports installing a few versions of Python 3 at a time, which means you can’t simply install the specific Python for your project to make a virtual environment in order to upgrade your Django project to 2.x. Not to mention that once brew updates python, given how brew installs Python, it could break your virtual environments pointing to an old python install if you specified the binary too specifically.

And that’s where PyEnv (github.com) comes in.

PyEnv is a set of scripts which help installing and automatically use the correct version of Python for your project.

It works by using some Shell Scripts to “shim” python, virtualenv, and etc. commands through its script which then uses the right version for you.

Once installed, running which python should point at your PyEnv script (not python). When you run python -V you should see the version pyenv selected. You might see the version from the python built-in to your system if you did not tell PyEnv to use a different version. That is the system python and typically you never want to rely on that version because:

1. It’s hard to change/upgrade
2. Apple has hinted they will remove python from the OS eventually
3. You could accidentally misconfigure it when the system is relying on it.

Installing PyEnv Link to heading

All you need to do to install PyEnv is clone their git repo into ~/.pyenv.

Then you update your path to look in PyEnv first like this:

export PATH="$HOME/.pyenv/bin:$PATH"

If you’re not using bash or zsh it may look different and you’ll need to look up how to manage your path in your specific shell.

Once you’ve updated your shell config you’ll want to create a new shell. You can use this little command to replace your current shell with a new copy.

exec $SHELL -l

Using PyEnv Link to heading

Let’s install Python 3.6.10

pyenv install 3.6.10

Now we can switch to it in our shell (this only affects our current terminal session)

pyenv shell 3.6.10

Check the version is what we expected. First check what pyenv has as the current python version and then check what version of python gets printed by Python itself.

pyenv shell
python -V

Now we can use pip to install the latest version of itself as well as setuptools and virtualenv

pip install -U pip setuptools virtualenv

Now we can use virtualenv to create a new environment for our project. I’ve often used this command

$(pyenv which virtualenv) -p $(pyenv which python) --prompt '(project)' .venv

The reason for using $(pyenv which ...) is to get absolute paths which I find more reliable.

This makes a folder .venv in your project which you can activate by running

source .venv/bin/activate

You can now manage your project dependencies in an isolated environment pointing at the specific version of python we selected.

Later on, if you setup this project years from now, you can use PyEnv to install that version of python again and dependencies you last used to get the project running again.