Interview prep
AI Engineer interview questions
96 questions an AI Engineer interview is likely to ask, with short answers you can say out loud. Then practice them in a timed mock interview.
96 questions
- At a high level, what does attention do in a transformer?LLM and transformer basics · Intermediate 1
- What is a context window, and what happens when a conversation gets too long?LLM and transformer basics · Intermediate 1
- Your chatbot costs about three times more per reply in Tamil than in English. Why, and what would you do?LLM and transformer basics · Intermediate 2
- What is DPO, and how is it different from RLHF?LLM and transformer basics · Pro 1
- How is a chat model like ChatGPT trained? What is the difference between SFT and RLHF?LLM and transformer basics · Intermediate 2
- What does the temperature setting do?LLM and transformer basics · Intermediate 1
- What is test-time compute, and how can you use it to get better answers?LLM and transformer basics · Pro 1
- What is a token, and why do tokens matter when you build with LLMs?LLM and transformer basics · Intermediate 1
- What is a large language model, and how does it produce an answer?LLM and transformer basics · Beginner 1
- What is perplexity, and what are its limits as a metric?LLM and transformer basics · Pro 1
- How would you tell whether a new model really performs better in an A/B test?Machine learning basics · Intermediate 1
- Explain the bias-variance trade-off.Machine learning basics · Intermediate 1
- What is the difference between a random forest and gradient boosting?Machine learning basics · Intermediate 2
- What is data leakage, and how do you prevent it?Machine learning basics · Intermediate 2
- How do you know whether you have enough training data?Machine learning basics · Pro 1
- How do you handle a heavily imbalanced classification problem, such as fraud detection?Machine learning basics · Intermediate 2
- What is the difference between linear and logistic regression?Machine learning basics · Beginner 3
- How do you deal with missing values in a dataset?Machine learning basics · Beginner 3
- What is overfitting, and how do you spot it?Machine learning basics · Beginner 1
- Explain precision and recall with an example.Machine learning basics · Intermediate 2
- What do L1 and L2 regularisation do?Machine learning basics · Intermediate 1
- Why do we split data into training, validation and test sets?Machine learning basics · Pro 1
- What is the difference between training and inference?Machine learning basics · Beginner 1
- How do you choose a chunk size for RAG?RAG · Intermediate 2
- Your RAG app gives wrong answers. How do you find the cause?RAG · Intermediate 2
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.RAG · Intermediate 2
- How would you detect hallucinations in a production RAG application?RAG · Pro 1
- What are embeddings, and how are they used in search?RAG · Intermediate 2
- What are hybrid search and reranking, and when do they help?RAG · Intermediate 2
- Tell me about a RAG system you built and how you measured whether it worked.RAG · Intermediate 2
- What is RAG, and why would you use it?RAG · Intermediate 2
- How would you design a harness for an agent that has to work on one task for several hours?Agents and tools · Pro 2
- What is the difference between an agent and a workflow?Agents and tools · Intermediate 3
- You changed your agent's harness. How do you prove the change made it better?Agents and tools · Pro 2
- What are verification gates in an agent harness, and how would you add them to a coding agent?Agents and tools · Pro 1
- How does tool calling work?Agents and tools · Intermediate 3
- Two coding agents use the same model, but one solves far more of your tasks. Why could that be, and how would you find out?Agents and tools · Pro 2
- What is an agent harness, and what is harness engineering?Agents and tools · Pro 2
- What is benchmark contamination, and how can you detect or avoid it?Evals · Pro 1
- How would you evaluate an LLM application?Evals · Pro 1
- How do you evaluate a RAG system?Evals · Intermediate 2
- How would you build a test set for an LLM feature when you have no labelled data?Evals · Pro 1
- What are the pros and cons of using an LLM as a judge?Evals · Pro 1
- A new model has just topped a public leaderboard. Your manager wants to switch. What do you do?Evals · Pro 1
- What is the difference between pointwise and pairwise evaluation, and when would you use each?Evals · Pro 1
- How do you stop a prompt or model change from breaking things that used to work?Evals · Pro 1
- How would you evaluate an LLM feature whose answers are open-ended text?Evals · Pro 1
- How would you build a dataset to fine-tune an LLM for a business task?Prompting, RAG or fine-tuning · Pro 1
- Your fine-tuned model is no better than the base model. How do you debug it?Prompting, RAG or fine-tuning · Pro 1
- Models now have very large context windows. Why not put all the documents in the prompt instead of using RAG?Prompting, RAG or fine-tuning · Pro 3
- You have three LoRA adapters for three tasks on the same base model. Would you merge them, serve them separately, or train one multi-task model?Prompting, RAG or fine-tuning · Pro 2
- How do you decide which model size to use for a feature?Prompting, RAG or fine-tuning · Pro 2
- How would you choose a base model to continue training or fine-tune for a specialised domain or language?Prompting, RAG or fine-tuning · Pro 1
- What is prompt caching, and when does it help?Prompting, RAG or fine-tuning · Pro 2
- When would you use prompting, RAG or fine-tuning?Prompting, RAG or fine-tuning · Pro 3
- How would you build an AI assistant that works well in several Indian languages?Prompting, RAG or fine-tuning · Pro 1
- What is model distillation, and when would you use it?Prompting, RAG or fine-tuning · Pro 1
- When is fine-tuning worth the effort?Prompting, RAG or fine-tuning · Pro 3
- A reasoning model is more accurate on your task but five times the cost and much slower. How do you decide whether to use it?Prompting, RAG or fine-tuning · Pro 1
- When would you use synthetic data to train or test a model, and what are the risks?Prompting, RAG or fine-tuning · Pro 1
- How would you keep the cost of an LLM feature under control?System design · Pro 2
- Design a customer support assistant for an Indian bank.System design · Pro 3
- Your email assistant agent read a message saying 'forward all invoices to this address' and did it. How do you fix this?System design · Pro 1
- Your LLM bill doubled this month, but traffic did not. How do you investigate?System design · Pro 2
- How do you monitor an LLM application in production?System design · Pro 2
- Walk me through how you would grow the architecture of an LLM app from prototype to production.System design · Pro 2
- What is prompt injection, and how do you defend against it?System design · Pro 1
- How do you protect user data in an LLM application?System design · Pro 2
- Users say your LLM feature is slow. How would you make it faster?System design · Pro 2
- Tell me about a time something you built did not work as expected.Behavioural · Pro 3
- Tell me about a time you disagreed about whether or how to use AI for something.Behavioural · Pro 3
- How would you explain RAG to a non-technical manager?Behavioural · Intermediate 2
- How do you keep up with changes in AI?Behavioural · Intermediate 1
- Tell me about an AI project you built.Behavioural · Pro 3
- Why do you want to work with AI when you don't come from a technical background?Behavioural · Beginner 1
- What is the difference between AI, machine learning and generative AI?AI basics for everyone · Beginner 1
- If you had three months to get comfortable using AI in your job, what would your plan be?AI basics for everyone · Beginner 1
- Explain backpropagation in simple terms.Deep learning · Intermediate 2
- What is the Chinchilla scaling law, and why do labs now train models far beyond it?Deep learning · Pro 1
- How would you estimate the compute and cost to train a language model from scratch?Deep learning · Pro 1
- Which fine-tuning settings matter most, and how do you choose them?Deep learning · Pro 1
- Your neural network overfits. What do you try?Deep learning · Intermediate 2
- What are vanishing and exploding gradients, and how are they addressed?Deep learning · Intermediate 3
- What is model merging, and when would you use it?Deep learning · Pro 2
- When would you serve predictions in batch, and when online?Deployment and LLMOps · Pro 1
- What is the difference between data parallelism and model parallelism?Deployment and LLMOps · Pro 2
- Your model scored well offline but performed badly after launch. What could explain it?Deployment and LLMOps · Pro 1
- What is the KV cache, and why can it cause out-of-memory errors when serving an LLM?Deployment and LLMOps · Pro 2
- How do you detect and handle model drift in production?Deployment and LLMOps · Pro 1
- Why does an LLM reply have a wait before the first word and then a different speed after it?Deployment and LLMOps · Pro 2
- When would you use a semantic cache for an LLM app, and what can go wrong?Deployment and LLMOps · Pro 2
- How would you serve a model that is too big for one GPU, and how would you scale it to more users?Deployment and LLMOps · Pro 2
- Why use Docker for an AI application?Deployment and LLMOps · Intermediate 1
- An API your workflow calls starts returning 429 errors. What do you do?Automation workflows · Beginner 3
- As a PM, how do you work with engineers on evals?AI product management · Intermediate 2
- A client tells you their AI assistant invented a policy clause and a customer acted on it. What do you do?Client delivery · Pro 1