Manage Your Conda Environment

At a Glance

Conda is an open-source tool that makes it easy to create, save, load, and switch between environments on your virtual machine. Each HISE IDE comes with a set of preinstalled tools and scientific libraries. The specific package depends on the modality you choose. Each modality is defined by key scientists based on specified project needs, such as scRNA-seq or flow cytometry (for details, see Manage NextGen IDE Packages). Sometimes, however, you might need to install specialized packages. To maintain reproducibility, this environment can be captured and shared with other scientists. This document shows you how to create and share a customized Conda environment.

Conda Commands

Use conda, as opposed to Pip, to install desired packages. Pip and Conda both use dependency management systems, and conda is chosen to be the default package management tool. Using pip could overwrite conda-installed packages, and will install packages from PyPi, which may not be fully compatible with Conda’s dependencies. Use conda, as opposed to pip, to install packages.

The following table summarizes common Conda commands, including some of those discussed in this document:

Conda Command	Example or note	Description
`conda info`	`conda info --envs`	Verify installation or version
`conda install <packagename>`	`conda install -c conda-forge r-seurat=4.3.0`	Install a package
`conda update conda`	`conda update conda -c conda-forge -n base -y --no-pin`	Update Conda to the latest version
`conda update <packagename>`	`conda update -c bioconda bwa`	Update a package
`conda config`	`conda config --add channels bioconda`	Modify configuration values in `.condarc`
`conda list`	`conda list -n bioenv`	List installed packages in the specified environment
`conda create`	`conda create -n bioinfo -c bioconda bwa`	Create a new conda environment from a list of specified packagesj
`conda activate`	`conda activate /home/workspace/environment/my_conda_env`	Activate the specified Conda environment
`conda search`	`conda search -c conda-forge r-seurat`	Search for packages and show related information
`conda remove --all`	`conda remove -n myenv --all`	Remove all packages and the Conda environment itself
`conda env list`	An asterisk appears next to the currently active environment	Lists all Conda environment. Same as `conda info --envs`

Instructions

Open a HISE IDE

1. Navigate to HISE, and use your organizational email address to sign in.

2. From the top navigation menu, click RESEARCH, and choose IDE NextGen from the drop-down menu.

3. On the IDE NextGen page, click Create IDE instance. (Alternatively, to open an existing instance, click on the card that represents the instance you want to open.)

4. When the IDE is ready, click the arrow to open it.

Create a custom kernel

A kernel is the computational engine of your Jupyter Notebook. By default, Jupyter uses the kernel associated with the environment you're working in. To test Conda packages that aren't part of your default Conda environment, you can create a custom kernel. You can then switch back and forth between the default kernel and the custome kernel. Before you create a new kernel, run `conda activate` to make sure the default conda environment is activated. Then new kernel will be based on the package configuration of this default kernel.

1. With your IDE open, click the Terminal icon.

2. To create a new Jupyter kernel, enter the following command for either Python or R:

Python

python -m ipykernel install --user --name=python_custom_kernel

Rscript -e "IRkernel::installspec(name = 'R_custom_kernel', displayname = ‘prera-il27’)"

3. Close the terminal window.

Open a Jupyter Notebook

1. The kernel you just created is now visible in the Launcher. To open a notebook in the new kernel, select the icon. You now

You now have a notebook open in your custom kernel. The kernel name appears in the upper-right corner of the notebook.

Search for new packages

You can usually pull packages directly from the Conda default channel. Sometimes you need to search community repositories like conda-forge. CompBio maintains a list of useful Conda packages and where to find them.

1. To search for packages, click the + sign, and then click the Terminal icon (see Step 2.1).

2. To search for bioinformatics packages, use the following command to search the bioconda channel:

conda config --add channels bioconda

3. Use the conda search command, followed by the arguments of your choice, as in the following example:

conda search -c conda-forge r-seurat

Install the new packages

Install with Conda (preferred)

1. To install the new packages, use conda install followed by the necessary arguments, as in the following example.

conda install -c conda-forge r-seurat=4.3.0

2. At the prompt, enter y to proceed.

After the package is executed, you should see a verification message like this one:

3. (Optional) If you're asked to update Conda, as in the above screenshot, do so now.

conda update -n base -c conda-forge conda

Install with pip (for packages not available through Conda or housed in GitHub)

1. Use the `pip install` command to install the package with pip, as in the following example:

pip install scanpy

2. A warning a message prompts you to confirm that you want to use Pip. To continue, click y.

3. To ensure that pip doesn't overwrite Conda-installed packages and cause conflicts in your environment, run the following command for Python or R:

Python

hp.conda_env_builds()

R

condaEnvBuilds()

Activate the new environment

1. Activate your environment by specifying either the name or the path, as in the following example:

conda activate /home/workspace/environment/my_conda_environment

NOTE

You can (and should) switch environments when necessary—if there are package conflicts or your use case changes, for example, or if you simply want to refocus your efforts.

Export and share your environment

You can use a command-line operation to export your Conda environment as a YAML file. This file contains all the original channels, packages, versions, and tags, as well as all associated dependencies. You can use the file to re-create your environment elsewhere, to back up your preferred setup, or to share your environment with researchers who want to reproduce or build on your work.

1. To export your environment, enter the following command:

conda env export > /home/workspace/environment/exported_env.yml

2. To make sure the export was successful, open your /environment directory and verify that the file is in it.

3. To share or re-create the environment using the exported file, enter the following command:

conda env create -f exported_env.yml

Related Resources

Manage NextGen IDE Packages