Can you do retrieval for long documents without a vector database?
RAG · Pro 1
Short answer
Yes. For long, well-structured documents such as annual reports or contracts, you can build a tree of the document's sections, like a table of contents with summaries, and let a model navigate it and open the relevant parts, as PageIndex does. Keyword search such as BM25 is another vector-free option. These can beat chunk similarity when structure matters.
Deeper answer
Vector search splits documents into chunks that lose context, such as which section a clause belongs to. Tree navigation keeps that structure and gives explainable paths: you can show which sections the model read.
The trade-offs are more model calls per query, slower answers and dependence on good document structure. For large collections of short documents, vectors or hybrid search usually remain the better choice.
What a strong answer covers
- Yes, for long structured documents
- Build a tree of sections with summaries
- Model navigates and opens relevant parts
- Keyword search like BM25 is another option
- Trade-offs: more calls, latency, needs structure
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- What kinds of documents suit tree navigation?
- What are the costs compared with vector search?
Common mistakes
- Assuming RAG always needs embeddings
- Ignoring latency of multi-step navigation
Learn it
Long documents without vectors
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- What are hybrid search and reranking, and when do they help?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- What are query rewriting and HyDE, and when do they help?
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?
- Tell me about a RAG system you built and how you measured whether it worked.
- What is RAG, and why would you use it?
- When would you use graph RAG instead of standard vector RAG?