How would you design a harness for an agent that has to work on one task for several hours?
Agents and tools · Pro 2
Short answer
Keep state outside the context window. An initializer writes a feature list and progress file; each session does one item, runs the checks, commits to git and updates the progress file. Reset or compact context between sessions, checkpoint so it can resume after a crash, and set time and cost budgets with clear points to ask a person.
Deeper answer
A long run fails in predictable ways: the context fills up and the agent loses the goal, it tries to do everything at once, it marks work done that does not work, or it crashes and starts over.
A common answer is a planner, generator and evaluator split. A planner or initializer turns the goal into a list of small features, for example as JSON where each item records whether it passes yet, plus a progress file. Each working session reads the progress file and git log, picks one unfinished feature, builds it, and runs tests or an evaluator agent before marking it done. It then commits and writes a short note for the next session.
Between sessions you either compact the context (summarise and continue) or reset it (start fresh and rebuild from files on disk). A reset works well only if the state on disk is complete. Checkpoints or durable execution let the run resume after a crash instead of starting again. Time and cost budgets cap the run, and the harness should stop and ask a person when checks keep failing, the budget is near its end or the next step is risky.
What a strong answer covers
- Plans the work into small items, such as a feature list
- Keeps state on disk: progress file and git
- Verifies each item before marking it done
- Manages context with resets or compaction
- Checkpoints, budgets and stopping to ask a person
Answer it out loud in the mock interview about 2.5 min
Follow-ups to expect
- When would you choose a context reset over compaction?
- Why keep the feature list as JSON rather than free text?
- What should make a long run stop and ask a person?
Common mistakes
- Relying on one huge context window instead of state on disk
- Letting the agent mark features done without a check
- No budget or no way to resume after a crash
Learn it
Harnesses for long-running agents
- What are common failure modes of agents, and how do you guard against them?
- What kinds of memory can an agent have, and how do you implement them?
- Your agent ran overnight and ran up a large API bill. What probably happened, and what do you change?
- What is the difference between an agent and a workflow?
- What are computer-use or browser agents good for, and what are their risks?
- What is context engineering, and why does it matter for agents?
- An agent completes a task correctly most of the time but sometimes goes badly wrong. How do you debug it?
- What makes a good tool for an agent?
- Design an agent that handles refund requests for an online store.
- You changed your agent's harness. How do you prove the change made it better?
- What are verification gates in an agent harness, and how would you add them to a coding agent?
- How does tool calling work?
- When should an AI agent ask a human for approval before acting?
- Explain state, nodes, edges and checkpoints in LangGraph.
- What are planning and reflection in agents, and when do they help?
- Two coding agents use the same model, but one solves far more of your tasks. Why could that be, and how would you find out?
- What is an agent harness, and what is harness engineering?
- What is the Model Context Protocol, and why does it matter?
- When is a multi-agent system worth it, and what are the risks?