Neraya GPUs-la models-a odurathu: replicas, tensor, pipeline parallelism
Oru model-a GPUs-ukkulla eppadi parappuranga-nu explain pannunga, serving illa training-ku adha copy panradhukkum (replicas), split panradhukkum (tensor illa pipeline parallelism) idaiyila choose pannunga, oru deployment-a size pannunga.
Pro 2 · Production engineerabout 2.5 hours7 sections
Short-aa sonna
Model oru GPU-la porundhina, adhoda copies-a odi neraya users-a serve pannunga. Porundhaadhuna, split pannunga: ovvoru layer-aiyum GPUs-ukkulla (tensor) illa layers-oda groups-a GPUs-ukkulla (pipeline).
Idhu yen mukkiyam
70B parameters-um adhukku mela irukkura open models oru GPU-la porundhaadhu, real traffic-ku oru copy-a vida adhigam thevai. Oru model-a eppadi parappanum-nu choose panradhu cost, speed, reliability-a decide pannum, adhu ML platform interviews-la oru standard kelvi.
Sign up free to read the whole lesson
The rest of this lesson: 7 parts, video lessons, the best course, practice and a quick check. It is free, takes a few seconds and needs no card.
Accounts are for learners aged 18 and over. Under 18? The quick answer and the first part of every lesson stay open: browse lessons by topic.
Lessons on AI Engineer0 of 70 done
- 1What AI, machine learning and a language model are
- 2AI careers in India
- 3Choosing your path
- 4Set up Python and run your first file
- 5Variables, loops and functions in Python
- 6Write your own functions
- 7Lists and dictionaries
- 8Files and errors
- 9The terminal and Git
- 10Packages and virtual environments
- 11Call a web API and read JSON
- 12Tables with pandas
- 13Maths for machine learning
- 14Statistics for machine learning
- 15Regression
- 16Classification
- 17Trees and gradient boosting
- 18Cross-validation and leakage
- 19Choosing a metric
- 20Neural networks
- 21PyTorch
- 22Transformers
- 23How a chat model is made: pre-training, SFT and preference tuning
- 24AI in Indian languages: tokens, quality gaps and what to use
- 25Model size and scaling laws: parameters, training data and compute
- 26Perplexity and cross-entropy: how language models are measured
- 27Call an LLM API from code
- 28System prompts and prompt design
- 29Tokens and the context window
- 30How a model picks its next word: temperature, top-p and top-k
- 31Get structured JSON output
- 32Errors, retries and cost
- 33Embeddings
- 34Chunk documents for retrieval
- 35Vector search
- 36Build a RAG app
- 37Check whether retrieval works
- 38Tool calling
- 39The agent loop
- 40Test sets and evals
- 41LLM judges and their limits
- 42Choosing a model: benchmarks, leaderboards and your own tests
- 43Scoring open-ended answers: exact checks, similarity and side-by-side comparison
- 44Catching hallucinations: why models make things up and how to detect it
- 45Regression tests for prompts
- 46Defend against prompt injection
- 47Docker and Compose
- 48REST APIs with FastAPI
- 49Serving a model
- 50Deploy and monitor an LLM app
- 51Monitoring and drift
- 52Caching and speed
- 53Inside LLM inference: prefill, decode and the KV cache
- 54Grow an AI app in layers: context, guardrails, routing, caching and agents
- 55Running models on many GPUs: replicas, tensor and pipeline parallelism
- 56Cost control
- 57Security and privacy
- 58Advanced harness engineering
- 59Harnesses for long-running agents
- 60How fine-tuning works: data, training and what can go wrong
- 61LoRA, QLoRA and the GPU memory maths
- 62Fine-tuning in practice: tools, settings and reading the loss curve
- 63Model merging: combining fine-tuned models into one
- 64Synthetic data and distillation: training with model-made data
- 65Building a training dataset: quality, coverage and quantity
- 66Reasoning models and test-time compute: when to let the model think longer
- 67Prompting, RAG or fine-tuning
- 68Design an AI system
- 69Quality, cost and speed trade-offs
- 70The design interview
Related
On these roadmaps: AI Engineer, Forward Deployed Engineer (FDE), MLOps Engineer
Deployment and opsPro 2: Production engineerVideo lessonsData parallelismTensor parallelismPipeline parallelismGPUvLLM