Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.
RAG · Intermediate 2
Short answer
Parse contracts carefully, keeping clause structure and tables, and store metadata such as client, contract type, dates and parties. Use hybrid search with filters and a reranker, answer only from retrieved clauses with citations, and say so when the answer is not found. Enforce per-matter access control, keep data in an approved region, and evaluate with lawyers' real questions.
Deeper answer
Legal work needs precision and trust. Chunk by clause rather than fixed size, keep the clause number and heading with each chunk, and store structured fields so a lawyer can say 'indemnity clauses in supplier contracts signed after 2024'. Exact terms matter in law, so keyword search alongside vectors helps.
Every answer should quote and link the clause it relies on, and say 'not found' rather than guess. Access control is essential: lawyers must see only matters they are cleared for. Build a test set with the firm's lawyers, measure retrieval and answer accuracy, and keep a human lawyer responsible for any advice.
What a strong answer covers
- Parses and chunks by clause, keeping structure
- Stores metadata and supports filters
- Uses hybrid search and reranking
- Answers with citations and refuses when not found
- Enforces access control per matter
- Evaluates with real lawyer questions
Answer it out loud in the mock interview about 2.5 min
Follow-ups to expect
- How would you handle scanned contracts?
- How would you measure whether it is good enough to use?
Common mistakes
- Fixed-size chunks that split clauses
- Ignoring which lawyer may see which matter
Learn it
- How do you choose a chunk size for RAG?
- Your RAG app gives wrong answers. How do you find the cause?
- How would you detect hallucinations in a production RAG application?
- What are embeddings, and how are they used in search?
- How do you make a RAG system cite sources and stay grounded?
- What are hybrid search and reranking, and when do they help?
- How do you keep a RAG index up to date as documents change?
- How do metadata filters improve retrieval?
- How would you build RAG over documents full of charts and images?
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?
- How do you make sure users only get answers from documents they are allowed to see?
- What are query rewriting and HyDE, and when do they help?
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?
- Tell me about a RAG system you built and how you measured whether it worked.
- Can you do retrieval for long documents without a vector database?
- What is RAG, and why would you use it?
- When would you use graph RAG instead of standard vector RAG?