Skip to main content

System design answer

Question answering over long legal contracts

Long context with prompt caching for one contract, retrieval across many, strict citations to clauses, and lawyer review of risky answers.
The question“Design a tool where a legal team uploads contracts and asks questions such as "what is the termination notice period?"”

1Clarify0-5 min

Ask before you draw. These answers change the design.

  • One contract at a time, or questions across hundreds?
  • How long are contracts, and are they scanned?
  • How exact must answers be, and who checks them?
  • Confidentiality rules: may text leave the region?

2Requirements and numbers5-8 min

It must

  • Upload and parse contracts with clause structure
  • Answer with the exact clause cited
  • Compare a clause across many contracts

And it must be

  • Every answer cites a clause; no citation, no answer
  • Documents stay in the firm's cloud region
  • Answer under 10 s for one contract

Numbers to say out loud

Contract size
40-120 pages, about 40,000-100,000 tokens
Library
5,000 contracts
Questions
2,000 a day

3High-level design8-12 min

  1. Upload → Parse with clause structure
  2. Parse with clause structure → Store clauses with numbers
  3. Question on one contract → fits the window → Whole contract, cached prefix
  4. Whole contract, cached prefix → Model
  5. Question across contracts → Retrieve matching clauses
  6. Retrieve matching clauses → Model
  7. Model → Answer citing clause numbers
  8. Answer citing clause numbers → Check quotes match
  9. Check quotes match → risky topic → Lawyer review

4Deep dives12-26 min

Long context or retrieval

For one contract, send it whole with the stable prefix cached, so follow-up questions are cheap and nothing is missed. Across the library, retrieve clauses by type (termination, liability) with metadata, then answer per contract in a table.

Citations

Clauses keep their numbers from parsing. The answer must quote the clause; code checks that the quote appears in that clause before showing it.

5Failures26-30 min

What can go wrongWhat stops it
Answer from the wrong clauseQuote check against the cited clause
Scanned contract with OCR errorsOCR confidence shown; low confidence goes to review
Confidential data leaves the regionModel in the firm's region through the gateway

6Evals and monitoring30-33 min

  • 200 questions over 20 contracts with lawyer-written answers
  • Citation accuracy and quote match rate

7Trade-offs33-35 min

  • Long context is simpler and more complete for one document but costs more; caching closes much of the gap
  • Requiring citations means some refusals, which lawyers prefer to guesses

Follow-ups to expect

  • How would you find every contract with an unusual liability cap?
  • What if two clauses contradict each other?

Practise out loud in the mock interview

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.