1ConceptIntermediate 1
Why use Docker for an AI application?
Short answer
Docker packages the app with its exact Python version, libraries and system dependencies into an image, so it runs the same on a laptop, in CI and in production. That ends 'works on my machine' problems, makes deployments repeatable and lets you run several services together with Docker Compose, such as an API, a vector database and a worker.
The explanation
Good practice: small base images, pinned dependency versions, a .dockerignore, no secrets baked into images, and a non-root user. Multi-stage builds keep images small.
For GPU work, images based on the CUDA runtime plus the NVIDIA container toolkit let containers use GPUs.
Our RAG service needed Python 3.12, a specific PyTorch version and a system library for PDF parsing. On a new teammate's laptop it failed for half a day. With a Dockerfile, everyone runs the same image, CI tests the same image, and production deploys it. Docker Compose starts the API, a Postgres database with pgvector and a background worker with one command.
They may follow up with
- What should never go into a Docker image?
- What is Docker Compose for?
Avoid
- Baking secrets into images
- Huge images with build tools left in