How do you protect user data in an LLM application?
System design · Pro 2
Short answer
Send the model only the data each task needs, keep API keys and secrets on the server, enforce access rules in code so users only retrieve their own data, mask personal data in logs, set retention limits, and check your provider's data use and retention terms. Follow local law, such as India's data protection rules.
Deeper answer
Data minimisation reduces harm if anything leaks. In RAG, filter documents by the user's permissions before retrieval, not after. Never put secrets in prompts, because system prompts can leak. Review provider terms on whether data is stored or used for training, and choose settings or plans that fit your obligations.
Also plan for user rights: export and deletion of their data. Logs and analytics are a common leak point, so treat them with the same care as the main database.
What a strong answer covers
- Sends only the data each task needs
- Keys and secrets stay on the server
- Access control enforced in code
- Masks personal data in logs and limits retention
- Checks provider terms and local law such as DPDP
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- Where in a RAG pipeline should permission checks happen?
- What would you check in an LLM provider's terms?
Common mistakes
- Relying on the model to refuse to show other users' data
- Logging raw prompts that contain personal data
Learn it
- How would you keep the cost of an LLM feature under control?
- Design a recommendation system for an Indian e-commerce app.
- Design a customer support assistant for an Indian bank.
- Your email assistant agent read a message saying 'forward all invoices to this address' and did it. How do you fix this?
- Your LLM bill doubled this month, but traffic did not. How do you investigate?
- How do you monitor an LLM application in production?
- Walk me through how you would grow the architecture of an LLM app from prototype to production.
- What is prompt injection, and how do you defend against it?
- Users say your LLM feature is slow. How would you make it faster?