LLM inference ke andar: prefill, decode aur KV cache
Samjhao ki LLM reply mein pehle shabd se pehle wait aur uske baad ek speed kyun hoti hai, KV cache ka size nikaalo, aur latency, throughput ya cost ke liye sahi lever chuno.
Pro 2 · Production engineerabout 3 hours7 sections
Seedha jawab
Model poora prompt ek saath padhta hai (prefill), phir ek-ek token likhta hai (decode), aur KV cache mein save kiya kaam reuse karta hai. Prefill pehle shabd ka wait set karta hai; decode uske baad ki speed.
Ye kyun zaroori hai
Ek AI product ke har speed aur cost decision, prompt length se lekar GPU size aur kaunsa API use karna hai tak, prefill aur decode ke kaam karne ke tareeke se nikalta hai. Jo engineers ise samajhte hain woh slow apps jaldi theek karte hain aur interviews mein serving ke sawaalon ke jawab confidence se dete hain.
Sign up free to read the whole lesson
The rest of this lesson: 7 parts, video lessons, the best course, practice and a quick check. It is free, takes a few seconds and needs no card.
Accounts are for learners aged 18 and over. Under 18? The quick answer and the first part of every lesson stay open: browse lessons by topic.
Lessons on AI Engineer0 of 70 done
- 1What AI, machine learning and a language model are
- 2AI careers in India
- 3Choosing your path
- 4Set up Python and run your first file
- 5Variables, loops and functions in Python
- 6Write your own functions
- 7Lists and dictionaries
- 8Files and errors
- 9The terminal and Git
- 10Packages and virtual environments
- 11Call a web API and read JSON
- 12Tables with pandas
- 13Maths for machine learning
- 14Statistics for machine learning
- 15Regression
- 16Classification
- 17Trees and gradient boosting
- 18Cross-validation and leakage
- 19Choosing a metric
- 20Neural networks
- 21PyTorch
- 22Transformers
- 23How a chat model is made: pre-training, SFT and preference tuning
- 24AI in Indian languages: tokens, quality gaps and what to use
- 25Model size and scaling laws: parameters, training data and compute
- 26Perplexity and cross-entropy: how language models are measured
- 27Call an LLM API from code
- 28System prompts and prompt design
- 29Tokens and the context window
- 30How a model picks its next word: temperature, top-p and top-k
- 31Get structured JSON output
- 32Errors, retries and cost
- 33Embeddings
- 34Chunk documents for retrieval
- 35Vector search
- 36Build a RAG app
- 37Check whether retrieval works
- 38Tool calling
- 39The agent loop
- 40Test sets and evals
- 41LLM judges and their limits
- 42Choosing a model: benchmarks, leaderboards and your own tests
- 43Scoring open-ended answers: exact checks, similarity and side-by-side comparison
- 44Catching hallucinations: why models make things up and how to detect it
- 45Regression tests for prompts
- 46Defend against prompt injection
- 47Docker and Compose
- 48REST APIs with FastAPI
- 49Serving a model
- 50Deploy and monitor an LLM app
- 51Monitoring and drift
- 52Caching and speed
- 53Inside LLM inference: prefill, decode and the KV cache
- 54Grow an AI app in layers: context, guardrails, routing, caching and agents
- 55Running models on many GPUs: replicas, tensor and pipeline parallelism
- 56Cost control
- 57Security and privacy
- 58Advanced harness engineering
- 59Harnesses for long-running agents
- 60How fine-tuning works: data, training and what can go wrong
- 61LoRA, QLoRA and the GPU memory maths
- 62Fine-tuning in practice: tools, settings and reading the loss curve
- 63Model merging: combining fine-tuned models into one
- 64Synthetic data and distillation: training with model-made data
- 65Building a training dataset: quality, coverage and quantity
- 66Reasoning models and test-time compute: when to let the model think longer
- 67Prompting, RAG or fine-tuning
- 68Design an AI system
- 69Quality, cost and speed trade-offs
- 70The design interview
Related
On these roadmaps: AI Engineer, Generative AI & LLM Engineer, Forward Deployed Engineer (FDE), MLOps Engineer
Deployment and opsPro 2: Production engineerVideo lessonsPrefillKV cacheTime to first token (TTFT)ThroughputSpeculative decodingContinuous batching