Skip to main content

Why do we split data into training, validation and test sets?

Machine learning basics · Pro 1

Short answer

To measure how a model or system performs on data it has not seen. You train on the training set, tune choices on the validation set, and report final performance once on the test set. If you tune on the test set, your score becomes optimistic and stops predicting real-world results.

Deeper answer

Each tuning decision leaks a little information about the data you tuned on. Keeping a test set untouched until the end gives an honest estimate. For time-based data, split by time so you do not train on the future.

The same discipline applies to LLM apps: keep a fixed eval set, and if you iterate heavily on it, keep a second held-out set to confirm the improvement is real. Make sure the sets reflect real user inputs, including messy and adversarial ones.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

Cross-validation and leakage

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.