What is RAG, and why would you use it?
RAG · Intermediate 2
Short answer
Retrieval-augmented generation fetches relevant text from your own documents and gives it to the model with the question, so it answers from that text. Use it when the model needs private, recent or frequently changing information, and when you want answers with sources you can check.
Deeper answer
A basic pipeline: split documents into chunks, embed them, store them in an index; at question time, embed the query, retrieve the closest chunks, and prompt the model to answer only from them and cite which chunk it used.
RAG beats fine-tuning for facts that change, because you update documents instead of retraining. It also allows access control (retrieve only what the user may see) and citations. Its weak point is retrieval: if the right chunk is not found, the answer will be wrong or missing, so you evaluate retrieval separately from generation.
What a strong answer covers
- Retrieves relevant text from your documents
- Gives it to the model with the question
- Use for private, recent or changing information
- Answers with sources you can check
Answer it out loud in the mock interview about 1 min
Follow-ups to expect
- How would you handle documents a user is not allowed to see?
- When would you not use RAG?
- How do you evaluate a RAG system?
Common mistakes
- Describing RAG as training the model on your documents
- Ignoring retrieval quality and only tuning the prompt
Learn it
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- What are hybrid search and reranking, and when do they help?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- What are query rewriting and HyDE, and when do they help?
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?
- Tell me about a RAG system you built and how you measured whether it worked.
- Can you do retrieval for long documents without a vector database?
- When would you use graph RAG instead of standard vector RAG?