Interview prep
MLOps Engineer interview questions
39 questions an MLOps Engineer interview is likely to ask, with short answers you can say out loud. Then practice them in a timed mock interview.
39 questions
- An agent completes a task correctly most of the time but sometimes goes badly wrong. How do you debug it?Agents and tools · Intermediate 3
- How would you evaluate an LLM application?Evals · Pro 1
- How would you build a test set for an LLM feature when you have no labelled data?Evals · Pro 1
- How do you stop a prompt or model change from breaking things that used to work?Evals · Pro 1
- What is prompt caching, and when does it help?Prompting, RAG or fine-tuning · Pro 2
- How would you keep the cost of an LLM feature under control?System design · Pro 2
- Your LLM bill doubled this month, but traffic did not. How do you investigate?System design · Pro 2
- How do you monitor an LLM application in production?System design · Pro 2
- How do you protect user data in an LLM application?System design · Pro 2
- Users say your LLM feature is slow. How would you make it faster?System design · Pro 2
- Tell me about an agent you built. What broke, and what did you learn?Behavioural · Intermediate 3
- Tell me about a production incident you handled.Behavioural · Pro 1
- When would you serve predictions in batch, and when online?Deployment and LLMOps · Pro 1
- How would you set up CI/CD for an LLM application?Deployment and LLMOps · Pro 1
- What is the difference between data parallelism and model parallelism?Deployment and LLMOps · Pro 2
- Why does experiment tracking matter, and what do you record?Deployment and LLMOps · Intermediate 3
- Your model scored well offline but performed badly after launch. What could explain it?Deployment and LLMOps · Pro 1
- Explain Kubernetes pods, deployments and services, and why you would use Kubernetes for AI apps.Deployment and LLMOps · Intermediate 3
- What is the KV cache, and why can it cause out-of-memory errors when serving an LLM?Deployment and LLMOps · Pro 2
- How would you load test an LLM feature and set a latency budget?Deployment and LLMOps · Pro 2
- Your model provider has an outage during peak hours. What should already be in place?Deployment and LLMOps · Pro 1
- How do you manage API keys and secrets across environments?Deployment and LLMOps · Intermediate 3
- How do you detect and handle model drift in production?Deployment and LLMOps · Pro 1
- Why does an LLM reply have a wait before the first word and then a different speed after it?Deployment and LLMOps · Pro 2
- What is quantisation, and what are the trade-offs?Deployment and LLMOps · Pro 1
- How would you serve an LLM feature as an API for a web app?Deployment and LLMOps · Intermediate 3
- How would you serve a model that is too big for one GPU, and how would you scale it to more users?Deployment and LLMOps · Pro 2
- How do you version prompts and models so you can roll back safely?Deployment and LLMOps · Pro 1
- What would you capture in traces for an LLM application?Deployment and LLMOps · Pro 1
- Why use Docker for an AI application?Deployment and LLMOps · Intermediate 1
- What does an LLM gateway do, and when do you need one?Deployment and LLMOps · Pro 1
- Why use vLLM to serve an open-weight model instead of a simple Hugging Face script?Deployment and LLMOps · Pro 1
- What should an audit trail for an AI system record, and who uses it?Responsible AI and governance · Pro 1
- Walk me through how you would assess the risks of a new AI feature before launch.Responsible AI and governance · Pro 1
- Your AI assistant gave harmful advice to a customer. What do you do?Responsible AI and governance · Pro 2
- Your company wants to buy an AI tool from a vendor. What do you check before approving it?Responsible AI and governance · Pro 1
- How does the EU AI Act sort AI systems by risk, and why could it matter to an Indian company?Responsible AI and governance · Pro 1
- What responsible AI checks would you run before launching an AI feature?Responsible AI and governance · Intermediate 3
- As a PM, how do you work with engineers on evals?AI product management · Intermediate 2