What is the difference between training and inference?
Machine learning basics · Beginner 1
Short answer
Training is when a model learns: its internal weights are adjusted using data, which is slow and expensive. Inference is when a trained model is used to make predictions or generate text, with the weights fixed. Most GenAI engineers work at inference time, calling a trained model through an API.
Deeper answer
Training a large language model needs huge datasets, many accelerators and weeks of time, so few companies do it from scratch. Fine-tuning is a smaller round of training on top of an existing model. Inference is each request a user makes; its cost is per call and depends on input and output size.
This split explains common interview points: the model does not learn from your chats during inference unless the provider later trains on data, its knowledge stops at the training cut-off, and the cheapest way to change behaviour is usually prompting or retrieval at inference time rather than training.
What a strong answer covers
- Training adjusts weights from data
- Training is slow and expensive
- Inference uses fixed weights to predict or generate
- Most app builders work at inference through an API
Answer it out loud in the mock interview about 1 min
Follow-ups to expect
- Does a model learn from the prompts you send it?
- Where does fine-tuning sit between the two?
Common mistakes
- Saying the model updates itself after every conversation
- Confusing fine-tuning with prompting
Learn it
What AI, machine learning and a language model are
- How would you tell whether a new model really performs better in an A/B test?
- Explain the bias-variance trade-off.
- What is the difference between a random forest and gradient boosting?
- How does k-means work, and how do you choose k?
- What is data leakage, and how do you prevent it?
- Give an example of feature engineering that made a model better.
- How do you know whether you have enough training data?
- How do you handle a heavily imbalanced classification problem, such as fraud detection?
- What is the difference between linear and logistic regression?
- How do you deal with missing values in a dataset?
- Walk me through an ML project you did, end to end.
- What is overfitting, and how do you spot it?
- Explain precision and recall with an example.
- What do L1 and L2 regularisation do?
- What is the difference between supervised and unsupervised learning?
- Why do we split data into training, validation and test sets?