Skip to main content

What is the difference between training and inference?

Machine learning basics · Beginner 1

Short answer

Training is when a model learns: its internal weights are adjusted using data, which is slow and expensive. Inference is when a trained model is used to make predictions or generate text, with the weights fixed. Most GenAI engineers work at inference time, calling a trained model through an API.

Deeper answer

Training a large language model needs huge datasets, many accelerators and weeks of time, so few companies do it from scratch. Fine-tuning is a smaller round of training on top of an existing model. Inference is each request a user makes; its cost is per call and depends on input and output size.

This split explains common interview points: the model does not learn from your chats during inference unless the provider later trains on data, its knowledge stops at the training cut-off, and the cheapest way to change behaviour is usually prompting or retrieval at inference time rather than training.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

What AI, machine learning and a language model are

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.