Two coding agents use the same model, but one solves far more of your tasks. Why could that be, and how would you find out?
Agents and tools · Pro 2
Short answer
The difference is probably the harness, not the model. Check the tools and their descriptions, what context each agent sees, whether it runs tests before finishing, how it handles long tasks and when it stops. Run both on the same fixed task set, read the traces of failed tasks, and change one harness part at a time.
Deeper answer
With the model fixed, everything else is harness. Common gaps in the weaker agent are: too many tools or vague tool descriptions, so it picks the wrong one; a context window filled with old tool output, so it loses the goal; no verification, so it stops at the first answer that looks right; no progress file, so it forgets earlier work on long tasks; and stop rules that end the run too early or let it loop.
To find out, do not guess. Build or reuse a fixed task set with automatic checks. Run both agents with the same model, settings and budget. For each failure, read the trace and label the cause: wrong tool, lost context, no check, gave up, ran out of budget. The most common label tells you which harness part to change first. Then change only that part, rerun the same tasks and compare pass rate, cost and time. Run each task more than once, because agent runs vary.
What a strong answer covers
- Says the harness, not the model, likely explains the gap
- Names likely harness gaps such as tools, context, verification or stop rules
- Uses a fixed task set with the same model and settings
- Reads traces of failures and labels causes
- Changes one part at a time and compares pass rate, cost and time
Answer it out loud in the mock interview about 2 min
Follow-ups to expect
- How would you label failures in the traces?
- Why change only one harness part at a time?
- What if the difference disappears when you rerun the tasks?
Common mistakes
- Assuming the model must be different or 'tuned'
- Comparing on a few hand-picked demos
- Changing several harness parts at once, so you cannot tell which one helped
Learn it
- What are common failure modes of agents, and how do you guard against them?
- What kinds of memory can an agent have, and how do you implement them?
- Your agent ran overnight and ran up a large API bill. What probably happened, and what do you change?
- How would you design a harness for an agent that has to work on one task for several hours?
- What is the difference between an agent and a workflow?
- What are computer-use or browser agents good for, and what are their risks?
- What is context engineering, and why does it matter for agents?
- An agent completes a task correctly most of the time but sometimes goes badly wrong. How do you debug it?
- What makes a good tool for an agent?
- Design an agent that handles refund requests for an online store.
- You changed your agent's harness. How do you prove the change made it better?
- What are verification gates in an agent harness, and how would you add them to a coding agent?
- How does tool calling work?
- When should an AI agent ask a human for approval before acting?
- Explain state, nodes, edges and checkpoints in LangGraph.
- What are planning and reflection in agents, and when do they help?
- What is an agent harness, and what is harness engineering?
- What is the Model Context Protocol, and why does it matter?
- When is a multi-agent system worth it, and what are the risks?