Interview prep
Generative AI & LLM Engineer interview questions
86 questions a Generative AI & LLM Engineer interview is likely to ask, with short answers you can say out loud. Then practice them in a timed mock interview.
86 questions
- At a high level, what does attention do in a transformer?LLM and transformer basics · Intermediate 1
- What is a context window, and what happens when a conversation gets too long?LLM and transformer basics · Intermediate 1
- Your chatbot costs about three times more per reply in Tamil than in English. Why, and what would you do?LLM and transformer basics · Intermediate 2
- What is DPO, and how is it different from RLHF?LLM and transformer basics · Pro 1
- How is a chat model like ChatGPT trained? What is the difference between SFT and RLHF?LLM and transformer basics · Intermediate 2
- What does the temperature setting do?LLM and transformer basics · Intermediate 1
- What is test-time compute, and how can you use it to get better answers?LLM and transformer basics · Pro 1
- What is a token, and why do tokens matter when you build with LLMs?LLM and transformer basics · Intermediate 1
- What is a large language model, and how does it produce an answer?LLM and transformer basics · Beginner 1
- Why do language models hallucinate, and how do you reduce it in an app?LLM and transformer basics · Beginner 1
- How do you know whether you have enough training data?Machine learning basics · Pro 1
- How do you deal with missing values in a dataset?Machine learning basics · Beginner 3
- What is the difference between training and inference?Machine learning basics · Beginner 1
- How do you choose a chunk size for RAG?RAG · Intermediate 2
- Your RAG app gives wrong answers. How do you find the cause?RAG · Intermediate 2
- Design a RAG system that lets a law firm search and ask questions about 50,000 contracts.RAG · Intermediate 2
- How would you detect hallucinations in a production RAG application?RAG · Pro 1
- What are embeddings, and how are they used in search?RAG · Intermediate 2
- How do you make a RAG system cite sources and stay grounded?RAG · Intermediate 3
- What are hybrid search and reranking, and when do they help?RAG · Intermediate 2
- How do you keep a RAG index up to date as documents change?RAG · Pro 1
- How do metadata filters improve retrieval?RAG · Intermediate 2
- How would you build RAG over documents full of charts and images?RAG · Pro 1
- Your RAG system must handle scanned PDFs, tables and slides. How do you ingest them?RAG · Intermediate 2
- How do you make sure users only get answers from documents they are allowed to see?RAG · Pro 1
- What are query rewriting and HyDE, and when do they help?RAG · Intermediate 3
- Your HR assistant keeps quoting last year's leave policy. How do you fix it?RAG · Intermediate 2
- Tell me about a RAG system you built and how you measured whether it worked.RAG · Intermediate 2
- Can you do retrieval for long documents without a vector database?RAG · Pro 1
- What is RAG, and why would you use it?RAG · Intermediate 2
- When would you use graph RAG instead of standard vector RAG?RAG · Pro 1
- What are common failure modes of agents, and how do you guard against them?Agents and tools · Intermediate 3
- How would you design a harness for an agent that has to work on one task for several hours?Agents and tools · Pro 2
- What is the difference between an agent and a workflow?Agents and tools · Intermediate 3
- An agent completes a task correctly most of the time but sometimes goes badly wrong. How do you debug it?Agents and tools · Intermediate 3
- You changed your agent's harness. How do you prove the change made it better?Agents and tools · Pro 2
- What are verification gates in an agent harness, and how would you add them to a coding agent?Agents and tools · Pro 1
- How does tool calling work?Agents and tools · Intermediate 3
- When should an AI agent ask a human for approval before acting?Agents and tools · Intermediate 3
- Two coding agents use the same model, but one solves far more of your tasks. Why could that be, and how would you find out?Agents and tools · Pro 2
- What is an agent harness, and what is harness engineering?Agents and tools · Pro 2
- What is benchmark contamination, and how can you detect or avoid it?Evals · Pro 1
- How would you evaluate an LLM application?Evals · Pro 1
- How do you evaluate a RAG system?Evals · Intermediate 2
- How would you build a test set for an LLM feature when you have no labelled data?Evals · Pro 1
- What are the pros and cons of using an LLM as a judge?Evals · Pro 1
- A new model has just topped a public leaderboard. Your manager wants to switch. What do you do?Evals · Pro 1
- What is the difference between pointwise and pairwise evaluation, and when would you use each?Evals · Pro 1
- How do you stop a prompt or model change from breaking things that used to work?Evals · Pro 1
- How would you evaluate an LLM feature whose answers are open-ended text?Evals · Pro 1
- How would you build a dataset to fine-tune an LLM for a business task?Prompting, RAG or fine-tuning · Pro 1
- Your fine-tuned model is no better than the base model. How do you debug it?Prompting, RAG or fine-tuning · Pro 1
- Models now have very large context windows. Why not put all the documents in the prompt instead of using RAG?Prompting, RAG or fine-tuning · Pro 3
- How do you decide which model size to use for a feature?Prompting, RAG or fine-tuning · Pro 2
- What is prompt caching, and when does it help?Prompting, RAG or fine-tuning · Pro 2
- When would you use prompting, RAG or fine-tuning?Prompting, RAG or fine-tuning · Pro 3
- How would you build an AI assistant that works well in several Indian languages?Prompting, RAG or fine-tuning · Pro 1
- What is model distillation, and when would you use it?Prompting, RAG or fine-tuning · Pro 1
- When is fine-tuning worth the effort?Prompting, RAG or fine-tuning · Pro 3
- A reasoning model is more accurate on your task but five times the cost and much slower. How do you decide whether to use it?Prompting, RAG or fine-tuning · Pro 1
- When would you use synthetic data to train or test a model, and what are the risks?Prompting, RAG or fine-tuning · Pro 1
- How would you keep the cost of an LLM feature under control?System design · Pro 2
- Design a customer support assistant for an Indian bank.System design · Pro 3
- Your email assistant agent read a message saying 'forward all invoices to this address' and did it. How do you fix this?System design · Pro 1
- Your LLM bill doubled this month, but traffic did not. How do you investigate?System design · Pro 2
- How do you monitor an LLM application in production?System design · Pro 2
- Walk me through how you would grow the architecture of an LLM app from prototype to production.System design · Pro 2
- What is prompt injection, and how do you defend against it?System design · Pro 1
- How do you protect user data in an LLM application?System design · Pro 2
- Users say your LLM feature is slow. How would you make it faster?System design · Pro 2
- Tell me about an agent you built. What broke, and what did you learn?Behavioural · Intermediate 3
- Tell me about a time something you built did not work as expected.Behavioural · Pro 3
- Tell me about a time you disagreed about whether or how to use AI for something.Behavioural · Pro 3
- How would you explain RAG to a non-technical manager?Behavioural · Intermediate 2
- How do you keep up with changes in AI?Behavioural · Intermediate 1
- Tell me about an AI project you built.Behavioural · Pro 3
- What is the difference between AI, machine learning and generative AI?AI basics for everyone · Beginner 1
- Your team asks you to suggest one place to start using AI. What would you pick, and why?AI basics for everyone · Beginner 1
- Which fine-tuning settings matter most, and how do you choose them?Deep learning · Pro 1
- What is the KV cache, and why can it cause out-of-memory errors when serving an LLM?Deployment and LLMOps · Pro 2
- Why does an LLM reply have a wait before the first word and then a different speed after it?Deployment and LLMOps · Pro 2
- When would you use a semantic cache for an LLM app, and what can go wrong?Deployment and LLMOps · Pro 2
- What information should you never paste into a public AI chatbot at work?Responsible AI and governance · Beginner 1
- An API your workflow calls starts returning 429 errors. What do you do?Automation workflows · Beginner 3
- As a PM, how do you work with engineers on evals?AI product management · Intermediate 2
- A client tells you their AI assistant invented a policy clause and a customer acted on it. What do you do?Client delivery · Pro 1