Cross-validation-ம் leakage-உம்
Model performance-ஐ நேர்மையா estimate பண்ணலாம்: train, validation, test splits, k-fold, time-based cross-validation, grouped splits; data leakage-ஐ எப்படி கண்டுபிடிச்சு, pipelines வச்சு தடுக்கிறதுன்னும்.
Intermediate 1 · LLM builderabout 3 hours7 sections
சுருக்கமா சொன்னா
Cross-validation, model train ஆகாத data-ல அதை test பண்ணும், பெரும்பாலும் k folds-ல. Leakage-னா எதிர்காலம் அல்லது target பத்தின information training-குள்ள நழுவி, scores பொய்யா நல்லா தெரியுறது.
இது ஏன் முக்கியம்
ML models production-ல ஏமாத்துறதுக்கு மிக common-ஆன காரணம், அவை நேர்மையில்லாம evaluate ஆனது தான். ஒரு notebook-ல 0.95 வாங்கி, உண்மையான உலகத்துல 0.70 வாங்குற model, கிட்டத்தட்ட எப்பவும் மோசமான split அல்லது leakage-ஆல பாதிக்கப்பட்டது. இதை சரியா பண்றது ஒரு core ML engineering skill, interview-லயும் அடிக்கடி வர்ற topic.
Sign up free to read the whole lesson
The rest of this lesson: 7 parts, video lessons, the best course, practice and a quick check. It is free, takes a few seconds and needs no card.
Accounts are for learners aged 18 and over. Under 18? The quick answer and the first part of every lesson stay open: browse lessons by topic.
Lessons on AI Engineer0 of 70 done
- 1What AI, machine learning and a language model are
- 2AI careers in India
- 3Choosing your path
- 4Set up Python and run your first file
- 5Variables, loops and functions in Python
- 6Write your own functions
- 7Lists and dictionaries
- 8Files and errors
- 9The terminal and Git
- 10Packages and virtual environments
- 11Call a web API and read JSON
- 12Tables with pandas
- 13Maths for machine learning
- 14Statistics for machine learning
- 15Regression
- 16Classification
- 17Trees and gradient boosting
- 18Cross-validation and leakage
- 19Choosing a metric
- 20Neural networks
- 21PyTorch
- 22Transformers
- 23How a chat model is made: pre-training, SFT and preference tuning
- 24AI in Indian languages: tokens, quality gaps and what to use
- 25Model size and scaling laws: parameters, training data and compute
- 26Perplexity and cross-entropy: how language models are measured
- 27Call an LLM API from code
- 28System prompts and prompt design
- 29Tokens and the context window
- 30How a model picks its next word: temperature, top-p and top-k
- 31Get structured JSON output
- 32Errors, retries and cost
- 33Embeddings
- 34Chunk documents for retrieval
- 35Vector search
- 36Build a RAG app
- 37Check whether retrieval works
- 38Tool calling
- 39The agent loop
- 40Test sets and evals
- 41LLM judges and their limits
- 42Choosing a model: benchmarks, leaderboards and your own tests
- 43Scoring open-ended answers: exact checks, similarity and side-by-side comparison
- 44Catching hallucinations: why models make things up and how to detect it
- 45Regression tests for prompts
- 46Defend against prompt injection
- 47Docker and Compose
- 48REST APIs with FastAPI
- 49Serving a model
- 50Deploy and monitor an LLM app
- 51Monitoring and drift
- 52Caching and speed
- 53Inside LLM inference: prefill, decode and the KV cache
- 54Grow an AI app in layers: context, guardrails, routing, caching and agents
- 55Running models on many GPUs: replicas, tensor and pipeline parallelism
- 56Cost control
- 57Security and privacy
- 58Advanced harness engineering
- 59Harnesses for long-running agents
- 60How fine-tuning works: data, training and what can go wrong
- 61LoRA, QLoRA and the GPU memory maths
- 62Fine-tuning in practice: tools, settings and reading the loss curve
- 63Model merging: combining fine-tuned models into one
- 64Synthetic data and distillation: training with model-made data
- 65Building a training dataset: quality, coverage and quantity
- 66Reasoning models and test-time compute: when to let the model think longer
- 67Prompting, RAG or fine-tuning
- 68Design an AI system
- 69Quality, cost and speed trade-offs
- 70The design interview
Related
On these roadmaps: AI Engineer, Machine Learning Engineer
Machine learningIntermediate 1: LLM builderVideo lessonsCross-validationData leakageOverfittingFeature engineering