uv, conda, Docker — choosing the right level of isolation for your workflow.
Every project gets its own Python environment. Never install packages globally. Never share an environment between projects. The cost of setup is ten minutes; the cost of skipping it is six months of cryptic version conflicts.
Two projects, both needing different versions of torch or numpy. Without isolation: one breaks. With isolation: both work. The same applies to Python versions, CUDA versions, system libraries.
The modern recommendation: use uv (a fast, sane replacement for pip/virtualenv/poetry, written in Rust). conda/mamba is the classic alternative; useful when you need non-Python dependencies. For deployment, wrap in a Docker container.
# With uv — fast, modern, recommended
uv init my-project
cd my-project
uv add torch numpy
uv run python train.py # automatically uses the project env
# Or with the classic approach
python -m venv .venv
source .venv/bin/activate
pip install torch numpy
The choice has two axes. Python-only or polyglot? (pip / uv handle Python; conda handles non-Python like CUDA, MKL.) Local development or production deployment? (Local: env manager. Production: container.)
uv (recommended for new projects) is a drop-in replacement for pip + virtualenv + pip-tools + poetry, written in Rust. ~10-100× faster than pip. Manages Python versions too (replaces pyenv). Single tool, no shell activation needed. If you're starting fresh, use this.
pip + venv: the bare-bones option. Use pip-tools (or pip freeze) to produce a lockfile from your requirements.in. Fine for simple projects; uv just makes the same workflow faster.
conda / mamba: when you need non-Python dependencies — CUDA toolkit, system libraries, MKL, R, etc. Heavier than pip but invaluable when the alternative is "install CUDA system-wide". mamba is the fast C++ implementation; almost always use it instead of conda directly.
poetry: was the modern choice before uv. Still solid; uv is faster and more focused. New projects: pick uv. Existing poetry projects: migrate when you have time.
Docker: for deployment and CI, not local development. Start FROM python:3.11-slim, install your locked dependencies, copy your code, set the entrypoint. The container is the unit of "deployable artefact" — even your training jobs should run in one for production work.
Lockfiles matter. pyproject.toml says "I want torch ≥ 2.0"; the lockfile (uv.lock, poetry.lock) records "I got torch 2.4.1, numpy 1.26.4, ...". Commit the lockfile. CI builds from the lockfile. This is what makes "reproducible environment" actually true.
# uv workflow — modern Python project from zero
uv init my-project --python 3.11
cd my-project
# Add dependencies
uv add "torch>=2.0" hydra-core wandb
uv add --dev pytest ruff mypy
# Install (creates .venv/, writes uv.lock)
uv sync
# Run scripts
uv run python scripts/train.py +experiment=exp01
# Update dependencies
uv lock --upgrade-package torch
# Run a tool without adding it
uv run --with ipython python
For production ML, you need reproducible builds from source to deployed container. That means pinned base images, locked dependencies, and CI that builds the artefact deterministically — not just docker build with a :latest base.
Base image discipline. Pin a specific digest: FROM python:3.11-slim@sha256:abcdef…, not FROM python:3.11. The 3.11 tag rolls forward when the upstream image is rebuilt; the digest doesn't. For GPU work: nvidia/cuda images at a specific tag, or PyTorch's official pytorch/pytorch.
Multi-stage builds. A "builder" stage installs build tools and compiles wheels; a "runtime" stage copies only the installed packages + your code. Final image is smaller and has fewer attack surfaces. Essential for production deployments.
Dependency hell on GPU stacks. CUDA version, cuDNN, NCCL, PyTorch, and the kernel driver all need to be compatible. Use PyTorch's published wheel matrix as ground truth, or install via conda/mamba which pulls compatible CUDA runtimes. Common failure: PyTorch built for CUDA 12.1 on a host with CUDA 11.8 driver.
Caching strategies for fast iteration. Layer order in Dockerfile matters: copy pyproject.toml + uv.lock first, install deps, then copy source. Only the source layer rebuilds on code changes — deps stay cached. With uv, pass --mount=type=cache,target=/root/.cache/uv in BuildKit to cache across builds.
Editable installs vs. wheels. Local dev: editable (pip install -e . or uv pip install -e .) so code changes are picked up without reinstall. Production: build a wheel and install it. Don't ship editable installs to production — they leak source paths and assume a writable filesystem layout.
Vendoring vs. dynamic resolution. For maximum reproducibility (think regulated ML), vendor wheels into a private registry. Your build never touches PyPI. Heavier but bulletproof — no "wheel withdrawn from PyPI" surprises.
# Multi-stage Dockerfile for a uv-based ML project
FROM python:3.11-slim AS builder
# Install uv
RUN pip install --no-cache-dir uv==0.4.20
WORKDIR /app
COPY pyproject.toml uv.lock ./
# Install deps into a venv at /app/.venv (fast with uv)
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-install-project
COPY src/ ./src/
RUN uv sync --frozen
# ---------- Runtime stage ----------
FROM python:3.11-slim AS runtime
# Copy only the resolved environment + code
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/src /app/src
ENV PATH="/app/.venv/bin:$PATH" PYTHONPATH="/app/src"
ENTRYPOINT ["python", "-m", "my_project.cli"]