When would you use prompting, RAG or fine-tuning?
Prompting, RAG or fine-tuning · Pro 3
Short answer
Start with prompting: it is fastest and cheapest to change. Add RAG when the model needs knowledge it lacks, especially private or changing information. Consider fine-tuning when you need a consistent behaviour, style or format that prompting cannot reach, have many good examples, and can evaluate the result.
Deeper answer
The approaches solve different problems. RAG adds knowledge at question time and keeps it current. Fine-tuning changes behaviour: tone, output format, a narrow skill, or getting a smaller, cheaper model to match a bigger one on one task. It is a poor way to add facts that change, because you must retrain to update them.
They also combine: a fine-tuned model can still use RAG. A strong answer mentions evaluation first (you cannot choose without a test set), the cost of maintaining training data, and the order: prompt, then retrieval, then fine-tuning, moving on only when measurements show the simpler step is not enough.
What a strong answer covers
- Start with prompting
- RAG for missing, private or changing knowledge
- Fine-tuning for consistent behaviour, style or format
- Needs many good examples and an eval
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- Can you combine RAG and fine-tuning? Give an example.
- What do you need before fine-tuning is worth trying?
Common mistakes
- Proposing fine-tuning to teach a model facts that change weekly
- Choosing an approach without mentioning how you would measure it
Learn it
- How would you build a dataset to fine-tune an LLM for a business task?
- Your fine-tuned model is no better than the base model. How do you debug it?
- Models now have very large context windows. Why not put all the documents in the prompt instead of using RAG?
- You have three LoRA adapters for three tasks on the same base model. Would you merge them, serve them separately, or train one multi-task model?
- How do you decide which model size to use for a feature?
- How would you choose a base model to continue training or fine-tune for a specialised domain or language?
- What is prompt caching, and when does it help?
- How would you build an AI assistant that works well in several Indian languages?
- What is model distillation, and when would you use it?
- When is fine-tuning worth the effort?
- A reasoning model is more accurate on your task but five times the cost and much slower. How do you decide whether to use it?
- When would you use synthetic data to train or test a model, and what are the risks?