What are hybrid search and reranking, and when do they help?
RAG · Intermediate 2
Short answer
Hybrid search combines vector search, which matches meaning, with keyword search such as BM25, which matches exact words like error codes and names. Reranking takes the top results and re-scores them with a stronger model to put the best first. Together they usually improve retrieval at some extra cost and latency.
Deeper answer
Vector search alone can miss exact identifiers; keyword search alone misses paraphrases. Running both and merging the results, for example with reciprocal rank fusion, covers both cases. A reranker, often a cross-encoder that reads the query and chunk together, is slower per item but more accurate, so it is applied only to the top few dozen candidates.
Published results, such as Anthropic's contextual retrieval write-up, show large drops in retrieval failures from combining these steps. Whether they pay off for you depends on your queries: measure recall before and after, and check the added latency fits your budget.
What a strong answer covers
- Vector search matches meaning
- Keyword search such as BM25 matches exact terms
- Reranking re-scores top results with a stronger model
- Trade-off: better retrieval for extra cost and latency
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- How would you merge results from two different searches?
- Why apply a reranker only to the top results?
Common mistakes
- Adding a reranker without measuring the latency cost
- Assuming vector search handles product codes well
Learn it
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- What are query rewriting and HyDE, and when do they help?
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?
- Tell me about a RAG system you built and how you measured whether it worked.
- Can you do retrieval for long documents without a vector database?
- What is RAG, and why would you use it?
- When would you use graph RAG instead of standard vector RAG?