Home When using env.yml with conda whats the difference between dependencies and pip dependencies?

Questions

When using env.yml with conda whats the difference between dependencies and pip dependencies?

November 4, 2022

I am creating a .SH script for setting up automatically my dev environment in Azure ML, according to this:

https://learn.microsoft.com/en-gb/azure/machine-learning/how-to-customize-compute-instance

The script looks like this:

#!/bin/bash
 
set -e
# https://pypi.org/project/azure-ai-ml/ 
# Requires: Python <4.0, >=3.7
# This script creates a custom conda environment and kernel based on a sample yml file.

conda env create  python=3.10
#conda env create -f env.yml

echo "Activating new conda environment"
conda activate envname
conda install -y ipykernel
echo "Installing kernel"
sudo -u azureuser -i <<'EOF'
conda activate envname
python -m ipykernel install --user --name envname --display-name "mykernelp310v2"
echo "Conda environment setup successfully."
pip install azure-ai-ml
EOF

my env looks like this:

name: p310v2

dependencies:
  - python=3.10
  - numpy
  - matplotlib
  - pandas
  - scikit-learn
  - pip:
       -kaggle==1.5

When I check this document:

https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/04-sharing-environments/index.html

I am confused between the dependencies section and the pip section. For example scikit-learn I could put in dependencies but also on the pip section, so whats the deal here?

>Solution :

dependencies are conda dependencies; listing them is equivalent to run conda install <deps>. For example conda installs numpy from https://anaconda.org/conda-forge/numpy .

pip means pip dependencies for pip install command; pip installs them from PyPI.org. For example, https://pypi.org/project/numpy/

The same for scikit-learn: https://anaconda.org/search?q=scikit-learn and https://pypi.org/search/?q=scikit-learn