Level 7 of 9
Pro 1: Quality engineer
At the end you can: Design test sets and evals; know the limits of LLM judges; defend against basic prompt injection.
About 25 hours in all; the steps written so far take about 53.5.
Steps
- Test sets and evalsBuild a test set for an LLM app and score every change against it.Not started · about 5 hours
- LLM judges and their limitsUse a model to grade answers, and check that its grades agree with a person's.Not started · about 3 hours
- Choosing a model: benchmarks, leaderboards and your own testsShortlist models by hard requirements, read benchmark scores and leaderboards with care, spot contamination, and make the final choice with your own test set.Not started · about 2.5 hours
Sign in or sign up to see the full roadmap
All 17 topics in order, with your progress saved on every device. It takes a few seconds, no card needed.
Accounts are for learners aged 18 and over. Under 18? Every lesson is still open: browse lessons by topic.
Level project
An eval harness that shows a measured improvement.