Design a customer support assistant for an Indian bank.
System design · Pro 3
Short answer
Clarify goals and limits first: languages, channels, which questions it may answer, and what must go to a human. Then: RAG over approved policy documents, read-only tools for account information behind strong authentication, strict guardrails against advice and data leaks, handoff to agents, evaluation before launch, and monitoring of quality, cost and escalations.
Deeper answer
Requirements: users ask in English, Hindi and other languages; answers must follow current policy; no account changes without verification; regulators expect privacy and auditability. Success metrics: resolution rate, correct escalation, customer satisfaction, cost per conversation.
Components: an intent router; retrieval over versioned policy documents with citations; tools for balance or transaction status with the user's verified identity, least privilege and full logging; a response check for forbidden content such as investment advice; human handoff with the conversation summary.
Risks: hallucinated policy, prompt injection through user messages, leaking another customer's data, and wrong answers in regional languages. Address each with grounding, permission checks in code (not in the prompt), and an eval set per language. Roll out to a small share of users, watch metrics, then expand.
What a strong answer covers
- Clarifies goals, languages, channels and scope first
- RAG over approved policy documents
- Account tools behind strong authentication
- Guardrails and handoff to a human
- Evaluates before launch and monitors after
Answer it out loud in the mock interview about 2 min
Follow-ups to expect
- How would you handle questions in Tamil if your documents are in English?
- What would you never let the bot do, and how would you enforce it?
- What metrics would you watch in the first week?
Common mistakes
- Jumping to model choice before clarifying requirements
- Enforcing data access rules only through the prompt
Learn it
- How would you keep the cost of an LLM feature under control?
- Design a recommendation system for an Indian e-commerce app.
- Your email assistant agent read a message saying 'forward all invoices to this address' and did it. How do you fix this?
- Your LLM bill doubled this month, but traffic did not. How do you investigate?
- How do you monitor an LLM application in production?
- Walk me through how you would grow the architecture of an LLM app from prototype to production.
- What is prompt injection, and how do you defend against it?
- How do you protect user data in an LLM application?
- Users say your LLM feature is slow. How would you make it faster?