Skip to main content

Topic roadmap

Deployment and ops

25 lessons, about 83.5 hours in all, from the basics to advanced. Start with the first, or jump to what you need.

Lessons, easiest first

  1. Deployment and ops · Intermediate 1 · about 3 hCI/CD with GitHub ActionsSet up a pipeline that tests every pull request, builds and pushes an image on merge, deploys to staging automatically and to production after approval, with a way to roll back.
  2. Deployment and ops · Intermediate 1 · about 4 hDocker and ComposePackage an AI service in a container image, run it with the right settings, and start a local stack of several services with Docker Compose.
  3. Deployment and ops · Intermediate 1 · about 6 hOne cloud in depthFind your way around one major cloud: regions, compute choices, storage, networking, identity and managed AI services, and estimate what a deployment will cost.
  4. Deployment and ops · Intermediate 2 · about 4 hEnvironments, secrets and TerraformKeep development, staging and production apart, manage secrets safely, and describe infrastructure as code with Terraform so it can be reviewed and rebuilt.
  5. Deployment and ops · Intermediate 2 · about 2 hExperiment tracking and model registryRecord every training run's code, data, parameters, metrics and model so results are comparable and reproducible, and manage model versions through a registry from staging to production.
  6. Deployment and ops · Intermediate 2 · about 3 hServing a modelPut a trained model into use, either as nightly batch scoring or as a real-time API, with consistent preprocessing, input validation, versioning, latency targets and safe rollout.
  7. Deployment and ops · Intermediate 2 · about 4 hServing an AI appRun an AI API in production: workers, timeouts, streaming responses, health checks and graceful shutdown.
  8. Deployment and ops · Intermediate 2 · about 3 hTracing LLM appsTrace every request through retrieval, prompts, model calls and tools, with inputs, outputs, tokens, cost and timings, and use traces to debug wrong answers.
  9. Deployment and ops · Intermediate 2 · about 2 hVersioning prompts and modelsTreat prompts, model choices and their settings as versioned, reviewed releases, so you always know what produced an answer and can roll back.
  10. Deployment and ops · Pro 1 · about 3 hDeployment modelsCompare the ways an AI system can be deployed, from a vendor's public API to the client's cloud, their own data centre and air-gapped sites, and match each to a client's data rules, budget and skills.
  11. Deployment and ops · Pro 1 · about 3 hGPUs and quantizationUnderstand what limits a model on a GPU (memory for weights and cache, compute and bandwidth), choose GPU types and counts, and use quantization to fit and speed up models.
  12. Deployment and ops · Pro 1 · about 5 hKubernetes and Helm basicsRead and change the Kubernetes set-up a client's platform team runs your service on: deployments, services, config, scaling, and Helm charts.
  13. Deployment and ops · Pro 1 · about 3 hLLM gatewaysPut a gateway in front of model providers to route requests, fall back when one fails, enforce rate limits and budgets per team, and track cost in one place.
  14. Deployment and ops · Pro 1 · about 3 hLoRA, QLoRA and the GPU memory mathsEstimate the GPU memory a model needs to run and to train, and explain why LoRA and QLoRA make fine-tuning fit on one card.
  15. Deployment and ops · Pro 1 · about 3 hMonitoring and driftMonitor a model in production: service health, input data drift, prediction drift, delayed ground truth and performance by segment, with alerts and a retraining policy.
  16. Deployment and ops · Pro 2 · about 3 hBatching and model tieringCut model costs without hurting quality: send easy requests to small models, escalate hard ones, and run work that can wait through discounted batch APIs.
  17. Deployment and ops · Pro 2 · about 2 hCaching and speedCut cost and waiting time with prompt caching, response caching and streaming.
  18. Deployment and ops · Pro 2 · about 2 hCost controlKnow where an app's money goes, and cut cost with model choice, batching and limits.
  19. Deployment and ops · Pro 2 · about 3 hCost of ownershipEstimate the full yearly cost of an AI system, comparing API, cloud-hosted and self-hosted options, including tokens, GPUs, infrastructure, people, compliance and change.
  20. Deployment and ops · Pro 2 · about 6 hDeploy and monitor an LLM appPut an LLM app online and watch its errors, latency, cost and answer quality.
  21. Deployment and ops · Pro 2 · about 3 hGrow an AI app in layers: context, guardrails, routing, caching and agentsStart an AI product as a single model call and add context, guardrails, a router and gateway, caches and agent steps only when a real need appears, with monitoring from day one.
  22. Deployment and ops · Pro 2 · about 3 hInside LLM inference: prefill, decode and the KV cacheExplain why an LLM reply has a wait before the first word and a speed after it, size a KV cache, and pick the right lever for latency, throughput or cost.
  23. Deployment and ops · Pro 2 · about 3 hLatency budgets and load testingSet a latency target per step, measure percentiles rather than averages, and load-test an AI service to find where it breaks before users do.
  24. Deployment and ops · Pro 2 · about 5 hOpen-weight models with vLLMServe an open-weight model on your own GPUs with vLLM: choose a model and GPU, size memory for weights and cache, use quantization, and expose an OpenAI-compatible API.
  25. Deployment and ops · Pro 2 · about 2.5 hRunning models on many GPUs: replicas, tensor and pipeline parallelismExplain how a model is spread across GPUs, choose between copying it (replicas) and splitting it (tensor or pipeline parallelism) for serving or training, and size a deployment.

On these role roadmaps

Interview questions on deployment and ops

Books on deployment and ops

All recommended books

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.