An agent completes a task correctly most of the time but sometimes goes badly wrong. How do you debug it?
Agents and tools · Intermediate 3
Short answer
Collect traces of the failing runs: every model call, tool call, argument, result and token count. Compare them with good runs to find the step where they diverge, usually a bad tool result, a misread instruction or missing context. Fix that step, then rerun a set of tasks to check the fix and catch regressions.
Deeper answer
Without traces you are guessing. Use structured logs or a tracing tool so each run is a tree of steps you can inspect. Group failures by cause: tool errors, wrong tool choice, poor planning, context overflow. Fixes differ: improve tool descriptions or error messages, add a validation step, shrink the context, or turn part of the agent into a fixed workflow.
Turn every real failure into a test case, so the eval set grows with production experience. Track success rate, steps and cost per task over time.
What a strong answer covers
- Collects traces of failing runs
- Compares with good runs to find where they diverge
- Finds the step: bad tool result, instruction or missing context
- Reruns a task set to check the fix and catch regressions
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- What would you log for each step?
- How do you turn production failures into tests?
Common mistakes
- Debugging by rerunning and hoping
- Logging only the final answer
Learn it
- What are common failure modes of agents, and how do you guard against them?
- What kinds of memory can an agent have, and how do you implement them?
- Your agent ran overnight and ran up a large API bill. What probably happened, and what do you change?
- How would you design a harness for an agent that has to work on one task for several hours?
- What is the difference between an agent and a workflow?
- What are computer-use or browser agents good for, and what are their risks?
- What is context engineering, and why does it matter for agents?
- What makes a good tool for an agent?
- Design an agent that handles refund requests for an online store.
- You changed your agent's harness. How do you prove the change made it better?
- What are verification gates in an agent harness, and how would you add them to a coding agent?
- How does tool calling work?
- When should an AI agent ask a human for approval before acting?
- Explain state, nodes, edges and checkpoints in LangGraph.
- What are planning and reflection in agents, and when do they help?
- Two coding agents use the same model, but one solves far more of your tasks. Why could that be, and how would you find out?
- What is an agent harness, and what is harness engineering?
- What is the Model Context Protocol, and why does it matter?
- When is a multi-agent system worth it, and what are the risks?