What are query rewriting and HyDE, and when do they help?
RAG · Intermediate 3
Short answer
Users ask short, vague or conversational questions that do not match how documents are written. Query rewriting has a model turn it into a clearer standalone query, or several variants. HyDE, hypothetical document embeddings, asks the model to write a fake answer and searches with its embedding, because an answer looks more like the documents than the question does.
Deeper answer
Rewriting is essential in chat, where 'what about for seniors?' only makes sense with history. Multi-query retrieval searches with several phrasings and merges the results, improving recall.
Both add a model call, so latency and cost go up, and a bad rewrite can drift from the user's intent. Measure on your eval set; for keyword-heavy queries such as error codes, hybrid search may help more.
What a strong answer covers
- Questions differ from how documents are written
- Rewriting creates a clearer standalone query or variants
- HyDE embeds a hypothetical answer
- Adds latency and cost
- Measure on an eval set
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- Why does HyDE work at all if the fake answer may be wrong?
- How would you rewrite a follow-up question in a chat?
Common mistakes
- Adding rewrites without measuring recall
- Ignoring conversation history
Learn it
Query rewriting, HyDE and parent-child chunks
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- What are hybrid search and reranking, and when do they help?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?
- Tell me about a RAG system you built and how you measured whether it worked.
- Can you do retrieval for long documents without a vector database?
- What is RAG, and why would you use it?
- When would you use graph RAG instead of standard vector RAG?