Skip to main content

System design answer

A customer support assistant for an Indian bank

RAG over approved policy documents, read-only account tools behind login, strict guardrails, and a clean hand-off to people.
The question“Design an AI assistant that answers customer questions for a bank, in English and Hindi, on the website and WhatsApp.”

1Clarify0-5 min

Ask before you draw. These answers change the design.

  • Which questions may it answer, and which must always go to a person (fraud, disputes, complaints)?
  • Which languages and channels, and how many conversations a day?
  • May it see account data, and how is the customer verified?
  • What does success mean: resolution rate, correct hand-offs, satisfaction, cost?

2Requirements and numbers5-8 min

It must

  • Answer product, fee and process questions from current policy, with links
  • Show balance and recent transactions after verification
  • Hand over to an agent with a summary
  • Work in English and Hindi on web and WhatsApp

And it must be

  • First words in under 2 s; full answer under 8 s
  • No investment advice; no data from another customer, ever
  • Every answer and action logged for audit
  • Data stays in an Indian region

Numbers to say out loud

Volume
50,000 conversations a day, peak 15 requests a second
Tokens per turn
About 3,000 in (prompt, history, 5 passages) and 300 out
Corpus
About 2,000 policy pages, updated weekly

3High-level design8-12 min

  1. Customer → Web chat or WhatsApp
  2. Web chat or WhatsApp → API
  3. API → Input guardrails
  4. Input guardrails → Router
  5. Router → policy question → Hybrid search + rerank
  6. Hybrid search + rerank → Policy index
  7. Router → account question, verified → Read-only account tools
  8. Router → fraud, dispute, unsure → Hand-off to an agent
  9. Hybrid search + rerank → Model via gateway
  10. Read-only account tools → Model via gateway
  11. Model via gateway → Output guardrails
  12. Output guardrails → Answer with citations

4Deep dives12-26 min

Retrieval

Chunk policies by heading with the title path, keep version and effective date as metadata, filter out superseded versions, and use hybrid search (codes like "NEFT" need keywords) with a reranker. Measure recall at 5 on 300 real questions in both languages.

Account tools and safety

Account tools are read-only and take the customer id from the verified session, never from the model. Permission checks live in code. Output checks block advice and any account number that is not the customer's own.

Languages

Keep one English index and translate the question for search, answering in the customer's language; build a separate eval set per language because quality differs.

5Failures26-30 min

What can go wrongWhat stops it
Hallucinated policyAnswer only from cited passages; refuse with a link to a person when nothing matches
Prompt injection in a messageTools are read-only and scoped to the session; untrusted text in marked sections
Model provider outageGateway fallback to a second region, then a scripted "we will call you back" flow
Stale policyWeekly re-index with effective dates; old versions filtered out

6Evals and monitoring30-33 min

  • Golden set of 300 questions per language with required facts and correct hand-off
  • Recall at 5 for retrieval, faithfulness by a calibrated judge
  • Online: hand-off rate, thumbs-down reasons, 50 traces reviewed a week

7Trade-offs33-35 min

  • RAG over fine-tuning: policies change weekly and answers need citations
  • More refusals early in exchange for zero invented policy
  • A small model for routing and a large one for answers, to control cost

Follow-ups to expect

  • How would you handle a question in Tamil?
  • What would you never let the bot do, and how is that enforced?
  • What do you watch in the first week after launch?

Practise out loud in the mock interview

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.