Skip to main content

What is model distillation, and when would you use it?

Prompting, RAG or fine-tuning · Pro 1

Short answer

Distillation trains a small student model to copy a large teacher model on a task, using the teacher's answers as training data. Use it when a large model does the task well but is too slow or costly at your volume, and the task is narrow enough for a small model to learn.

Deeper answer

The steps are: run the teacher on many real inputs, filter its outputs, fine-tune the student on those pairs, and test the student on real, hand-labelled data it has never seen. A good outcome is quality close to the teacher at a fraction of the cost and latency.

The limits matter. A student rarely beats its teacher and copies its mistakes. It can learn the teacher's style without its reasoning, producing confident wrong answers on harder or new inputs. Check the teacher provider's terms, since many forbid using outputs to train competing models.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

Synthetic data and distillation: training with model-made data

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.