Python Virtual Environments Tutorial

Introduction

This tutorial will walk through the basics of Python 3 virtual environments. The official documentation is available at https://docs.python.org/3/library/venv.html. All of these commands are run from your command prompt/shell/terminal, not from within Python.

What is a Python virtual environment

A virtual environment is a directory that contains its own installation of Python and its own set of libraries (site packages).

Why use it

Python virtual environments are particularly useful if you have multiple versions of Python installed, or you work on many projects that have different library requirements.

By default, without a virtual environment, all python and pip commands will use the default executables, usually your system install. It is better to keep your system Python clean of unnecessary site packages. Otherwise, over time, you will add lots of things to your system packages and things might conflict and cause problems. Using an isolated environment for each project ensures easy reproducability and reduced conflict.

Create a venv

To create a virtual environment, run in your command prompt or shell:

# Create a virtual environment named `my_venv` in your current directory
python3 -m venv my_venv

# Optionally inherit the system site package
python3 -m venv my_venv --system-site-packages

It can be useful at times to inherit the system site packages with the system-site-packages flag. This is particularly helpful if you have a library that is difficult to build or install, or is provided by another package provider (like a Linux apt or yum install). In these cases, inheriting the site packages may be necessary to avoid rebuilding libraries. In general, this is rare and I would avoid doing it unless you have a specific need to do so.

How to use a virtual environment

To use a virtual environment, you activate and deactivate them. This will modify your PATH environment variable.

Alternatively, you don't have to "activate" the environment, you can directly run the python interpreter using its full path. This is useful for running cron jobs. See the next examples.

Activate a virtual environment

In Linux/Mac:

my_venv/bin/activate

In Windows:

my_venv\Scripts\activate.bat

Once activate, you use python and pip like normal, except they will point to your virtual environment. When you activate a virtual environment, a few other things happen:

• python and pip in your PATH switches to the virtual environment Python
• Python will have access to the site packages from the virtual environment only
• pip packages and things installed with setup.py will go in your virtual environment's site packages
• Your PATH will be updated to include the Python virtual environment's script directory. Any executables installed through a package will be available in your shell path.

Run without activating

You can run a Python interpreter directly to avoid having to activate the virtual environment. This is useful for scripting or setting up cron jobs.

# Example of running a python virtual environment directly
/home/nanodano/venv/bin/python -m pip list

Deactivate a virtual environment

To "exit" a virtual environment, deactivate it.

deactivate

See which python and pip are being used

You can always check which python and pip your PATH environment variable is pointing to by using the following commands.

In Linux/Mac:

which pip
which python

In Windows:

where pip
where python

Find site packages directory

To see what directories are being used to search for packages, invoke the site module directly by running:

python -m site

Learn more about how Python searches for imports, how the PYTHONPATH environment variable is used, and how to use sys.path in my Python import, sys.path, and PYTHONPATH Tutorial.

Also check out my tutorial on how to import a package by name using a string variable.

View installed packages

To list the installed packages when a virtual environment is active, from the shell run:

pip list

Install packages to virtual environment

Once the virtual environment is activated (which you can confirm with which python/where python), you can install packages by using pip or running a setup.py file that uses distutils. The installed packages will go in to your virtual environment's site packages directory.

For example:

# Install using `pip` and pypi.org packages
python -m pip install flask
pip install flask  # Equivalent when virtual environment is activated

# Install from source
python setup.py install

You can also use a requirements.txt file to install a group of modules. This is useful for quickly setting up a new virtual environment.

# requirements.txt contains list of packages to install
# and can be created with `pip freeze > requirements.txt`
pip install -r requirements.txt

Important tips

• If Python is not finding a package that you are sure you installed, it is probably installed to a different instance/virtual environment. Make sure your IDE is using the right virtual environment and you installed the packages to the right environment. Use python -m pip list to check.
• python may refer to different version of python depending on whether you are using it in your command prompt/shell, pycharm, visual studio code, or another editor. Don't always assume python from different places is calling the same Python environment.
• Avoid using pip by itself. Using python -m pip will always guarantee you are using the pip associated with that specific python being called, instead of potentially calling a pip associated with a different python.
• Sometimes there is a pip3 to go with python3 to differentiate python 2 executables python/pip.
• You can call a python directly from path without having to activate venv /path/to/venv/python and it will use the site packages from the virtual environment this is particularly useful for cron jobs where you want to invoke python directly without the extra activating a virtual environment
• Sometimes Linux distributions require you to install virtualenv as a separate package. For example sudo apt install python3-virtualenv.
• I recommend using a separate virtual environment for each project.
• You should never copy or move around virtual environments. Always create new ones.
• Ignore the virtual environment directories from repositories. For example, .gitignore them.

Conclusion

After reading this you should have an idea of how to set up and use a basic virtual environment with Python 3.

References

• https://docs.python.org/3/library/venv.html
• Python import, sys.path, and PYTHONPATH Tutorial
• How to import a package by name using a string variable