How do you know whether you have enough training data?
Machine learning basics · Pro 1
Short answer
There is no fixed number; it depends on the task, base model and method. Start with 50 to 100 excellent examples and check they help. Then train on 25%, 50% and 100% of the data and score each on the same test set. A rising curve says collect more; a flat one says improve quality or coverage.
Deeper answer
Simple tasks and strong base models need less data; light methods like LoRA work with hundreds to thousands of examples, while full fine-tuning usually wants far more. Gains usually shrink as the set grows.
If a small set shows no gain at all, more of the same rarely fixes it: check the format, prompt and settings. Look at scores by segment too, since a flat overall curve can hide one weak group, such as a second language, that needs targeted examples.
What a strong answer covers
- Depends on task, base model and method
- Start small and check it helps
- Measures a learning curve on subsets
- Flat curve means improve quality or coverage
Answer it out loud in the mock interview about 1 min
Follow-ups to expect
- What would you do if the curve is flat but one segment is weak?
- How does the choice of LoRA or full fine-tuning change the answer?
Common mistakes
- Quoting a fixed number of examples
- Never measuring a learning curve
Learn it
Building a training dataset: quality, coverage and quantity
- How would you tell whether a new model really performs better in an A/B test?
- Explain the bias-variance trade-off.
- What is the difference between a random forest and gradient boosting?
- How does k-means work, and how do you choose k?
- What is data leakage, and how do you prevent it?
- Give an example of feature engineering that made a model better.
- How do you handle a heavily imbalanced classification problem, such as fraud detection?
- What is the difference between linear and logistic regression?
- How do you deal with missing values in a dataset?
- Walk me through an ML project you did, end to end.
- What is overfitting, and how do you spot it?
- Explain precision and recall with an example.
- What do L1 and L2 regularisation do?
- What is the difference between supervised and unsupervised learning?
- Why do we split data into training, validation and test sets?
- What is the difference between training and inference?