Virtual Environments and Packages in Python
Python applications will often use packages and modules that don’t come as part of the standard library. Applications will sometimes need a specific version of a library, because the application may require that a particular bug has been fixed or the application may be written using an obsolete version of the library’s interface.
This means it may not be possible for one Python installation to meet the requirements of every application. If application A needs version 1.0 of a particular module but application B needs version 2.0, then the requirements are in conflict and installing either version 1.0 or 2.0 will leave one application unable to run.
The solution for this problem is to create a virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
Different applications can then use different virtual environments. To resolve the earlier example of conflicting requirements, application A can have its own virtual environment with version 1.0 installed while application B has another virtual environment with version 2.0. If application B requires a library be upgraded to version 3.0, this will not affect application A’s environment.
Programs and other executable files can be in many directories, so operating systems provide a search path that lists the directories that the OS searches for executables.
The path is stored in an environment variable, which is a named string maintained by the operating system. This variable contains information available to the command shell and other programs.
The path variable is named as PATH in Unix or Path in Windows (Unix is case sensitive; Windows is not).
In Mac OS, the installer handles the path details. To invoke the Python interpreter from any particular directory, you must add the Python directory to your path.
Here are important environment variables, which can be recognized:
- PYTHONPATH - It has a role similar to PATH. This variable tells the Python interpreter where to locate the module files imported into a program. It should include the Python source library directory and the directories containing Python source code. PYTHONPATH is sometimes preset by the Python installer.
- PYTHONSTARTUP - It contains the path of an initialization file containing Python source code. It is executed every time you start the interpreter. It is named as .pythonrc.py in Unix and it contains commands that load utilities or modify PYTHONPATH.
- PYTHONCASEOK - It is used in Windows to instruct Python to find the first case-insensitive match in an import statement. Set this variable to any value to activate it.
- PYTHONHOME - It is an alternative module search path. It is usually embedded in the PYTHONSTARTUP or PYTHONPATH directories to make switching module libraries easy.
Creating Virtual Environments
The module used to create and manage virtual environments is called
venv will usually install the most recent version of Python that you have available. If you have multiple versions of Python on your system, you can select a specific Python version by running
python3 or whichever version you want.
To create a virtual environment, decide upon a directory where you want to place it, and run the
venv module as a script with the directory path:
python3 -m venv tutorial-env
This will create the
tutorial-env directory if it doesn’t exist, and also create directories inside it containing a copy of the Python interpreter, the standard library, and various supporting files.
Once you’ve created a virtual environment, you may activate it.
On Windows, run:
On Unix or MacOS, run:
(This script is written for the bash shell. If you use the csh or fish shells, there are alternate
activate.fish scripts you should use instead.)
Activating the virtual environment will change your shell’s prompt to show what virtual environment you’re using, and modify the environment so that running
python will get you that particular version and installation of Python. For example:
$ source ~/envs/tutorial-env/bin/activate (tutorial-env) $ python Python 3.5.1 (default, May 6 2016, 10:59:36) ... >>> import sys >>> sys.path ['', '/usr/local/lib/python35.zip', ..., '~/envs/tutorial-env/lib/python3.5/site-packages'] >>>
Managing Packages with pip
You can install, upgrade, and remove packages using a program called pip. By default
pip will install packages from the Python Package Index, <https://pypi.org>. You can browse the Python Package Index by going to it in your web browser, or you can use
pip’s limited search feature:
(tutorial-env) $ pip search astronomy skyfield - Elegant astronomy for Python gary - Galactic astronomy and gravitational dynamics. novas - The United States Naval Observatory NOVAS astronomy library astroobs - Provides astronomy ephemeris to plan telescope observations PyAstronomy - A collection of astronomy related tools for Python. ...
pip has a number of subcommands: “search”, “install”, “uninstall”, “freeze”, etc. (Consult the Installing Python Modules guide for complete documentation for
You can install the latest version of a package by specifying a package’s name:
(tutorial-env) $ pip install novas Collecting novas Downloading novas-22.214.171.124.tar.gz (136kB) Installing collected packages: novas Running setup.py install for novas Successfully installed novas-126.96.36.199
You can also install a specific version of a package by giving the package name followed by
== and the version number:
(tutorial-env) $ pip install requests==2.6.0 Collecting requests==2.6.0 Using cached requests-2.6.0-py2.py3-none-any.whl Installing collected packages: requests Successfully installed requests-2.6.0
If you re-run this command,
pip will notice that the requested version is already installed and do nothing. You can supply a different version number to get that version, or you can run
pip install --upgrade to upgrade the package to the latest version:
(tutorial-env) $ pip install --upgrade requests Collecting requests Installing collected packages: requests Found existing installation: requests 2.6.0 Uninstalling requests-2.6.0: Successfully uninstalled requests-2.6.0 Successfully installed requests-2.7.0
pip uninstall followed by one or more package names will remove the packages from the virtual environment.
pip show will display information about a particular package:
(tutorial-env) $ pip show requests --- Metadata-Version: 2.0 Name: requests Version: 2.7.0 Summary: Python HTTP for Humans. Home-page: http://python-requests.org Author: Kenneth Reitz Author-email: email@example.com License: Apache 2.0 Location: /Users/akuchling/envs/tutorial-env/lib/python3.4/site-packages Requires:
pip list will display all of the packages installed in the virtual environment:
(tutorial-env) $ pip list novas (188.8.131.52) numpy (1.9.2) pip (7.0.3) requests (2.7.0) setuptools (16.0)
pip freeze will produce a similar list of the installed packages, but the output uses the format that
pip install expects. A common convention is to put this list in a
(tutorial-env) $ pip freeze > requirements.txt (tutorial-env) $ cat requirements.txt novas==184.108.40.206 numpy==1.9.2 requests==2.7.0
requirements.txt can then be committed to version control and shipped as part of an application. Users can then install all the necessary packages with
(tutorial-env) $ pip install -r requirements.txt Collecting novas==220.127.116.11 (from -r requirements.txt (line 1)) ... Collecting numpy==1.9.2 (from -r requirements.txt (line 2)) ... Collecting requests==2.7.0 (from -r requirements.txt (line 3)) ... Installing collected packages: novas, numpy, requests Running setup.py install for novas Successfully installed novas-18.104.22.168 numpy-1.9.2 requests-2.7.0
pip has many more options. Consult the Installing Python Modules guide for complete documentation for
pip. When you’ve written a package and want to make it available on the Python Package Index, consult the Distributing Python Modules guide.