Your HR assistant keeps quoting last year's leave policy. How do you fix it?
RAG · Intermediate 2
Short answer
The old document is still in the index and is being retrieved. I would find it, then fix the pipeline: remove or archive replaced documents, store an effective date and version as metadata, filter retrieval to the current version by default, and re-index on every change. Then add test questions about the new policy so it cannot slip back.
Deeper answer
Stale answers are usually an ingestion problem, not a model problem. If old and new versions sit side by side, the retriever may prefer the old one simply because its wording matches the question better.
The durable fix is lifecycle management: each document has an owner, a version, an effective date and a status; the index updates when the source changes; and retrieval filters out superseded versions. Showing the source and its date in the answer lets employees spot problems, and evaluation questions on recent changes catch regressions.
What a strong answer covers
- Identifies that the old document is still indexed and retrieved
- Adds version or effective-date metadata
- Filters retrieval to current documents
- Updates the index when sources change
- Adds test questions for the change
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- How would you handle a policy that changes mid-year for some employees only?
- How do you know which documents are superseded?
Common mistakes
- Trying to fix it with a prompt telling the model to prefer new policies
- Deleting old files by hand without fixing the pipeline
Learn it
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- What are hybrid search and reranking, and when do they help?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- What are query rewriting and HyDE, and when do they help?
- Tell me about a RAG system you built and how you measured whether it worked.
- Can you do retrieval for long documents without a vector database?
- What is RAG, and why would you use it?
- When would you use graph RAG instead of standard vector RAG?