LoRA, QLoRA aur GPU memory ka maths
Andaaza lagao ki ek model ko chalane aur train karne ke liye kitni GPU memory chahiye, aur samjhao ki LoRA aur QLoRA fine-tuning ko ek card par kaise fit karte hain.
Pro 1 · Quality engineerabout 3 hours8 sections
Seedha jawab
Model ko apne weights ke liye memory chahiye, aur training ko kai guna zyada. LoRA weights ko freeze karke har layer ke do chhote matrices train karta hai, aur QLoRA frozen weights ko 4-bit mein bhi store karta hai.
Ye kyun zaroori hai
Model aapki GPU par fit hota hai ya nahi, ye cost, hardware aur aap kya bana sakte ho, sab tay karta hai. Interviewers aapse memory ka andaaza wahin par lagwate hain, aur clients poochte hain ki ek set-up doosre se das guna mehnga kyun hai.
Sign up free to read the whole lesson
The rest of this lesson: 8 parts, video lessons, the best course, practice and a quick check. It is free, takes a few seconds and needs no card.
Accounts are for learners aged 18 and over. Under 18? The quick answer and the first part of every lesson stay open: browse lessons by topic.
Lessons on AI Engineer0 of 70 done
- 1What AI, machine learning and a language model are
- 2AI careers in India
- 3Choosing your path
- 4Set up Python and run your first file
- 5Variables, loops and functions in Python
- 6Write your own functions
- 7Lists and dictionaries
- 8Files and errors
- 9The terminal and Git
- 10Packages and virtual environments
- 11Call a web API and read JSON
- 12Tables with pandas
- 13Maths for machine learning
- 14Statistics for machine learning
- 15Regression
- 16Classification
- 17Trees and gradient boosting
- 18Cross-validation and leakage
- 19Choosing a metric
- 20Neural networks
- 21PyTorch
- 22Transformers
- 23How a chat model is made: pre-training, SFT and preference tuning
- 24AI in Indian languages: tokens, quality gaps and what to use
- 25Model size and scaling laws: parameters, training data and compute
- 26Perplexity and cross-entropy: how language models are measured
- 27Call an LLM API from code
- 28System prompts and prompt design
- 29Tokens and the context window
- 30How a model picks its next word: temperature, top-p and top-k
- 31Get structured JSON output
- 32Errors, retries and cost
- 33Embeddings
- 34Chunk documents for retrieval
- 35Vector search
- 36Build a RAG app
- 37Check whether retrieval works
- 38Tool calling
- 39The agent loop
- 40Test sets and evals
- 41LLM judges and their limits
- 42Choosing a model: benchmarks, leaderboards and your own tests
- 43Scoring open-ended answers: exact checks, similarity and side-by-side comparison
- 44Catching hallucinations: why models make things up and how to detect it
- 45Regression tests for prompts
- 46Defend against prompt injection
- 47Docker and Compose
- 48REST APIs with FastAPI
- 49Serving a model
- 50Deploy and monitor an LLM app
- 51Monitoring and drift
- 52Caching and speed
- 53Inside LLM inference: prefill, decode and the KV cache
- 54Grow an AI app in layers: context, guardrails, routing, caching and agents
- 55Running models on many GPUs: replicas, tensor and pipeline parallelism
- 56Cost control
- 57Security and privacy
- 58Advanced harness engineering
- 59Harnesses for long-running agents
- 60How fine-tuning works: data, training and what can go wrong
- 61LoRA, QLoRA and the GPU memory maths
- 62Fine-tuning in practice: tools, settings and reading the loss curve
- 63Model merging: combining fine-tuned models into one
- 64Synthetic data and distillation: training with model-made data
- 65Building a training dataset: quality, coverage and quantity
- 66Reasoning models and test-time compute: when to let the model think longer
- 67Prompting, RAG or fine-tuning
- 68Design an AI system
- 69Quality, cost and speed trade-offs
- 70The design interview
Related
On these roadmaps: AI Engineer, Generative AI & LLM Engineer, Machine Learning Engineer, MLOps Engineer
Deployment and opsPro 1: Quality engineerVideo lessonsLoRA (low-rank adaptation)QLoRAQuantizationGPUWeights