What Is an Agent Harness? (And Why Your LLM Needs One)

The model is not the agent. The model is a function that maps text to text. Everything that makes it feel like an agent -- that it can read files, run commands, recover from a failed step, and stop when the job is done -- lives in the code wrapped around it. That wrapper is the agent harness, and it is where almost all of the engineering actually happens.

If your LLM project works in demos but falls apart on real tasks, the model is rarely the problem. The harness is.

What an agent harness actually is

An agent harness is the runtime loop and supporting machinery that turns a stateless chat model into something that can pursue a goal over many steps. At minimum it does five things:

• Runs the loop: call model, parse output, execute any tool calls, feed results back, repeat until done.
• Dispatches tools and returns their results in a format the model can use.
• Manages context: decides what goes into the prompt on each turn and what gets dropped or summarised.
• Verifies and recovers: checks results, handles errors, and prevents the model from running away.
• Enforces boundaries: permissions, sandboxing, and stopping conditions.

The model supplies reasoning. The harness supplies everything that makes that reasoning safe, grounded, and repeatable. Swap in a stronger model and a bad harness still produces an unreliable agent. Keep the model fixed and improve the harness, and the same model suddenly handles tasks it failed before.

The loop is the heart of it

Every agent harness is, underneath, a while loop:

messages = [system_prompt, user_goal]
while True:
    response = model.call(messages, tools=tool_schemas)
    if response.tool_calls:
        results = [run_tool(c) for c in response.tool_calls]
        messages.append(response)
        messages.extend(results)
    else:
        return response  # model produced a final answer

That is the whole skeleton. The interesting parts are the questions this loop forces you to answer: When do you stop? What if a tool throws? What if the model calls a tool that does not exist, or with garbage arguments? What if messages grows past the context window? A naive loop ignores all of these and works right up until it does not. A real harness has a deliberate answer to each.

Tool dispatch: the model's hands

Tools are how the agent touches the world -- read a file, query a database, call an API. The harness owns the contract between the model's intent to call a tool and the actual execution.

Three things matter here:

• Schema clarity. The tool description is prompt engineering, not documentation. A vague description produces wrong calls. Spell out when to use it, what each argument means, and what it returns.
• Result shaping. Tool output goes straight back into context, so it costs tokens and attention. Returning a 50,000-line log dump buries the signal. Return the relevant slice, or a summary plus a way to fetch more.
• Error handling. When a tool fails, the harness decides whether the model sees the raw error (so it can adapt) or a cleaned-up message. Often the model can recover from a clear error on its own -- if you let it.

Context management: what the model can actually see

A model only knows what is in its context window on the current turn. The harness decides what that is, and this is the single biggest lever on agent quality.

Left unmanaged, a long task fills the window with stale tool output and old reasoning until the important instructions fall off the edge or get drowned out. Good harnesses actively curate: they summarise finished sub-tasks, drop superseded file reads, keep the original goal pinned, and surface only what the next step needs. This is why two agents on the same model can behave completely differently -- one remembers the goal ten steps in, the other has forgotten why it started.

If you take one idea from this post, make it this: context is a budget you spend, not a bucket you fill.

Verification and recovery

The difference between a demo and a tool you trust is what happens when something goes wrong. A model will confidently claim a test passed that it never ran, or report a file written that it never wrote. A harness that takes the model's word for it inherits every hallucination.

Stronger harnesses bake in verification: run the test and check the exit code rather than believing the summary, diff the file rather than trusting the claim, require evidence before accepting "done". When a step fails, they feed the failure back so the model can correct course, with a cap on retries so a confused agent does not burn your budget in a loop. Verification is also where subagents earn their place -- spinning off a fresh context to check a result, free of the optimism that built it.

Boundaries: permissions and stopping

An agent that can run commands can run destructive commands. The harness is the only thing standing between "refactor this function" and "delete the repository because that technically resolved the failing test."

Boundaries come in a few forms: a permission layer that gates dangerous actions (often a human confirm), a sandbox that limits blast radius, and explicit stopping conditions so the loop ends -- on success, on a step cap, or on a cost ceiling. None of this is glamorous, and all of it is the reason you can leave the agent running without watching every token.

If you want to see a mature harness in practice, the coding agents are the clearest example. I wrote about using Claude Code as a daily pair -- much of what makes it useful is harness work: how it manages context across a long session, gates risky shell commands, and verifies its own changes rather than declaring victory.

Frameworks vs. building your own

You do not have to write the loop yourself; plenty of frameworks ship one. The tradeoff is the usual one. A framework gets you a working agent fast and hides the loop, the context strategy, and the tool plumbing. That is great until you hit a reliability problem -- and then the thing you most need to understand is exactly the thing the framework abstracted away.

My advice: write the bare loop once, by hand, even if you later adopt a framework. Twenty lines teaches you what every framework is doing and where its defaults will bite you. When you are running this at real volume, the harness also becomes an infrastructure question -- concurrency, retries, and cost -- which I get into in building a production LLM platform on Kubernetes.

Conclusion

An agent harness is not a detail you bolt on after picking a model -- it is the agent. The loop, tool dispatch, context management, verification, and boundaries are where reliability is won or lost, and they are almost entirely independent of which model you call. If your agent is flaky, resist the urge to swap models first. Instrument the harness: log what is in context on each turn, verify tool results instead of trusting them, and put a hard cap on the loop. Fix the harness and the same model will surprise you.

For more engineering write-ups, see the rest of the blog.

FAQ

Is an agent harness the same as an agent framework?

No. A framework (LangChain, the various agent SDKs) is one implementation of a harness, with opinions baked in. The harness is the concept: the loop and supporting machinery around the model. You can build a harness with a framework, or in 20 lines of your own code.

Do I need an agent harness if I only make single API calls?

No. If you send one prompt and use one response, there is no loop, no tool dispatch, and no state to manage -- so there is no harness. You need one the moment the model has to take multiple steps, use tools, or act on its own output.

What is the most common reason agents are unreliable?

Context management, more often than the model. As a task runs long, unmanaged context fills with stale output until the goal and key instructions get crowded out. Curating what the model sees each turn usually helps more than upgrading the model.

Why verify tool results if the model reports them?

Because models will confidently report results that never happened -- a test that "passed" but never ran. A harness that trusts those reports inherits every hallucination. Checking exit codes and diffs instead of summaries is what makes an agent trustworthy.

How do I stop an agent from looping forever?

Put stopping conditions in the harness: a step cap, a cost ceiling, and a clear success check. The model will not reliably decide to stop on its own, so the loop has to.