Skip to main content

Level 7 of 9

Pro 1: Quality engineer

At the end you can: Design test sets and evals; know the limits of LLM judges; defend against basic prompt injection.

About 25 hours in all; the steps written so far take about 53.5.

Steps

  1. Test sets and evalsBuild a test set for an LLM app and score every change against it.Not started · about 5 hours
  2. LLM judges and their limitsUse a model to grade answers, and check that its grades agree with a person's.Not started · about 3 hours
  3. Choosing a model: benchmarks, leaderboards and your own testsShortlist models by hard requirements, read benchmark scores and leaderboards with care, spot contamination, and make the final choice with your own test set.Not started · about 2.5 hours

Sign in or sign up to see the full roadmap

All 17 topics in order, with your progress saved on every device. It takes a few seconds, no card needed.

Accounts are for learners aged 18 and over. Under 18? Every lesson is still open: browse lessons by topic.

Level project

An eval harness that shows a measured improvement.

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.