Skip to main content

Your RAG app gives wrong answers. How do you find the cause?

RAG · Intermediate 2

Short answer

Separate retrieval from generation. For a set of failing questions, check whether the right chunk was retrieved. If not, fix retrieval: chunking, embeddings, query rewriting or hybrid search. If it was retrieved but the answer is still wrong, fix the prompt, context order or model. Measure each change on a test set.

Deeper answer

A practical sequence: collect failing examples; for each, log the retrieved chunks and see whether the answer was in them. Count how many failures are retrieval misses and how many are generation errors. Retrieval misses often come from chunks cut in the wrong place, missing metadata, vocabulary mismatch, or the document not being indexed at all.

Generation errors come from too much irrelevant context, instructions that do not force grounding, or the model ignoring the context. Build a small test set with the expected source for each question, track recall at k and answer correctness, and change one thing at a time.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

Check whether retrieval works

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.