Virtual Environments (Good)

library(reticulate)
use_python("/Users/chammika/miniconda3/envs/science-work/bin/python", required = TRUE)

What to expect in this chapter

Python allows you to maintain multiple environments, each with different packages and even different versions of Python. In this short chapter, I will discuss how to create and manage these environments and why they are beneficial. I will also touch on other things you can do to keep your Python environment updated and clean.

Why Use Virtual Environments?

1. Avoid Clutter: When you install a package using conda (or pip), it will typically install it to your default (base) installation. Installing packages globally like this can lead to a cluttered and unwieldy environment, especially when tools like conda or pip attempt to resolve dependencies.

2. Safe Testing: You might want to test a package safely without risking your base installation. Virtual environments allow experimentation without affecting the global Python setup.

3. Collaboration: If you are working on a collaborative project, sharing your code and environment setup ensures consistency among team members.

4. Version Conflicts: Some packages may have compatibility issues with specific versions. Isolating each project’s dependencies prevents breaking changes when upgrading or downgrading packages.

5. Legacy Code: There are instances when package upgrades will break your code due to code compatibility issues. Virtual environments allow you to maintain specific versions of packages to get around this problem.

Working with Environments

The following is a quick summary based on the instructions at the Conda wesbite. Please refer to that page for more details about optimizing the setup process.

If your installation is based on pip you must use venv. You can find more details about it here.

Create

Let me show you two ways to create a virtual environment. I recommend the second method, which uses a configuration file, as it is more reproducible and easier to share.

Manually

The following command creates a virtual environment named my_virtual_env that uses Python version 3.10. It also installs specific versions of SciPy and NumPy.

conda create --name my_virtual_env python=3.10 scipy=0.17.3 numpy=1.24

Using a File

You can also define the environment using a .yml file. I use this approach often to reinstall my environemnts and is recommended.

1. Create a .ymlfile. Lets call it my-environment.yml and add the following content:

   name: my_virtual_env
   channels:
    - conda-forge
    - defaults
   dependencies:
    - python=3.10
    - scipy=0.17.3
    - numpy=1.24
    - pip
    - pip:
        - tqdm

2. Use this file to create the environment from the file:

   conda env create --file my-environment.yml

Update packages

Using a yaml file offers a clean way to update and include or remove packages. You can use the following syntax:

conda env update --file environment.yml --prune

--prune removes dependencies no longer listed in the file.

Activate

To use an environment, you must activate it from the terminal. The following command activates the environment, switching to its specific Python and package setup.

conda activate my_virtual_env

Deactivate

Use this command to return to your base environment.

conda deactivate

Export/Share

Export the environment’s packages to a file for sharing:

conda env export > environment.yml

Others can replicate your environment by running:

conda env create --file environment.yml

List Environments

You can get a list of all the environment in your systme by using:

conda env list