Your fine-tuned model is no better than the base model. How do you debug it?
Prompting, RAG or fine-tuning · Pro 1
Short answer
First check the pipeline: can the model overfit 20 examples? If not, check data format, masking and the chat template. Then read the loss curves: flat means the learning rate is too low or labels are wrong; spikes mean it is too high; rising validation loss means overfitting. Finally, compare on a locked test set against a good prompt.
Deeper answer
I would read a sample of training examples for errors and inconsistency, and confirm the template used in training matches the one used at inference, a common silent failure. I would change one setting at a time: learning rate first, then epochs, then batch size with gradient accumulation if memory is tight.
If 50 to 100 good examples show no gain at all, more data rarely helps; the setup is usually wrong. If gains appear but lag in one segment, I would add data for that segment rather than tune further.
What a strong answer covers
- Sanity check: overfit a few examples
- Checks data format, masking and template
- Reads loss curves to diagnose
- Compares on a test set against a prompt baseline
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- What learning rate would you start with for LoRA?
- How do you choose the number of epochs?
Common mistakes
- Changing many settings at once
- Judging only by training loss
Learn it
Fine-tuning in practice: tools, settings and reading the loss curve
- How would you build a dataset to fine-tune an LLM for a business task?
- Models now have very large context windows. Why not put all the documents in the prompt instead of using RAG?
- You have three LoRA adapters for three tasks on the same base model. Would you merge them, serve them separately, or train one multi-task model?
- How do you decide which model size to use for a feature?
- How would you choose a base model to continue training or fine-tune for a specialised domain or language?
- What is prompt caching, and when does it help?
- When would you use prompting, RAG or fine-tuning?
- How would you build an AI assistant that works well in several Indian languages?
- What is model distillation, and when would you use it?
- When is fine-tuning worth the effort?
- A reasoning model is more accurate on your task but five times the cost and much slower. How do you decide whether to use it?
- When would you use synthetic data to train or test a model, and what are the risks?