Pattern · Retrieval and RAG
Hybrid search + rerank
Run keyword and vector search together, merge the results, then let a slower, sharper model reorder the top few.
Say it in the interview
For retrieval I would use hybrid search: BM25 for exact terms like policy numbers and vectors for meaning, merged with reciprocal rank fusion, then a cross-encoder reranks the top 40 down to 5. I would justify it with recall at 5 on a golden set before and after.
When to use it
- Queries mix meaning with exact terms: product codes, error ids, names
- Vector search alone misses exact matches
- You can afford 100-300 ms more for better precision
How it works
- Query → BM25 keyword search
- Query → Vector search
- BM25 keyword search → Reciprocal rank fusion
- Vector search → Reciprocal rank fusion
- Reciprocal rank fusion → top 30-50 → Cross-encoder reranker
- Cross-encoder reranker → top 5 → Prompt
Trade-offs
- Usually beats either search alone on real queries
- The reranker adds latency and cost per query
- Two indexes to keep in sync
Pitfalls interviewers probe
- Reranking too few candidates to matter
- Tuning on a handful of queries instead of a golden set