The Agent Harness: What Actually Turns an LLM Into an Agent

A client team once showed me their new "agent" and asked why it kept confidently reporting that a deployment succeeded when it hadn't touched the deploy script at all. They'd wired a capable model up to a shell tool and called it done. Nothing was wrong with the model. What was missing was everything else — the part that decides when the loop stops, what counts as evidence the task actually finished, and what the agent is and isn't allowed to touch along the way. That "everything else" has a name in this industry now: the harness. It's the least glamorous part of agent design and it's where most of the real engineering — and most of the actual failure modes — live.

I've written about the agent landscape broadly and about how you evaluate agent systems once they exist. This piece is about what you're actually building before there's anything to evaluate: the scaffolding that takes a model that can only do one thing — read a prompt, emit tokens — and turns it into something that can plan, act, observe the result, and keep going until a job is actually finished. Swap the model out from under a good harness and the agent usually still works, just a little worse. Swap a great model into a bad harness and you get exactly what that client had: fluent, confident, and wrong.

What is an agent harness?

The harness is the software system surrounding a language model that manages the loop of invocation, tool execution, and context assembly needed to accomplish a multi-step task — everything that isn't the model's weights. A raw model call is stateless and single-shot: you send a prompt, you get a completion, the conversation is over unless you resend the whole history yourself. An agent harness is what turns that into something that can run for minutes or hours, call tools, read their output, decide what to do next, and stop only when the task is actually done — or when it hits a limit the harness itself enforces.

This distinction matters practically because it changes where you spend your engineering time. Teams that treat "build an agent" as "pick the best model and give it some tools" consistently underperform teams that treat the harness as its own system with its own design decisions — because the harness is where reliability, safety, and cost control actually get decided. The model contributes reasoning quality. The harness contributes almost everything about whether that reasoning turns into a correct, safe, bounded outcome.

How does the core agentic loop actually work?

Strip away every product's branding and the loop underneath nearly every coding or ops agent shipping today — Claude Code, Cursor's agent mode, GitHub's agentic tooling, OpenAI's Codex CLI, Cognition's Devin — is a variation on the same shape: assemble context, call the model, inspect what it returned, act on it, feed the result back in, repeat.

graph TD
    CTX["Assemble context
system prompt + tool schemas + history"] --> MODEL["Model call
emits text or a tool-call request"]
    MODEL -->|"tool call requested"| PERM["Permission gate
allow / deny / ask user"]
    PERM -->|"denied"| MODEL
    PERM -->|"allowed"| EXEC["Execute tool
sandboxed"]
    EXEC --> VERIFY["Verification step
tests, lint, build, real output"]
    VERIFY --> APPEND["Append result to context"]
    APPEND --> COMPACT["Context compaction
if approaching window limit"]
    COMPACT --> CTX
    MODEL -->|"task complete signal"| DONE["Return to caller"]
          

The step people underrate is the permission gate sitting between "the model wants to do something" and "the thing actually happens." A model deciding to run rm -rf or push to a shared branch is not, by itself, a safety mechanism — it's a proposal. Whether that proposal executes is a decision the harness makes, and a well-designed harness makes it based on the specific action requested (read a file versus overwrite one, run a linter versus force-push), not a blanket trust level assigned to the whole session. This is a security boundary, not a courtesy layer for skittish users: giving an LLM a shell and a scratch directory is a materially larger attack surface than a chatbot ever was, and it needs to be treated with the same rigor as any other privilege boundary in a system, independent of how well-aligned the underlying model is.

Why does tool schema design matter more than people expect?

A tool definition is a structured description — name, parameters, a JSON Schema for arguments, a natural-language description of what it does and when to use it — that the harness exposes to the model so it can request an action instead of only emitting prose. The model doesn't call a function directly; it emits a structured request that names a tool and fills in arguments matching the schema, and the harness is the thing that actually executes whatever that tool maps to.

The part that's easy to get wrong: a tool's description and error messages are prompts, whether you think of them that way or not. A tool called run_command with a one-line description invites the model to use it as a catch-all, including for things a narrower, purpose-built tool would do more reliably and more safely. A tool that fails silently, or returns an unhelpful stack trace instead of a clear "the file doesn't exist at that path, did you mean X" message, teaches the model nothing useful about what to try next — it just burns a turn. I've watched teams spend weeks tuning prompts and system messages while their actual bottleneck was a handful of poorly-named, poorly-documented tools the model kept misusing. Tool design is prompt design; it just doesn't look like it because it lives in a schema instead of a paragraph.

How do you stop a long-running agent from drowning in its own history?

Context windows are finite, and an agent that runs for an hour generates far more tokens of tool output, file contents, and intermediate reasoning than any window holds. A harness that just keeps appending everything eventually hits the wall mid-task, and what happens next is the difference between a merely limited agent and a broken one: does it silently truncate the oldest, possibly load-bearing instructions, or does it manage the transition deliberately?

Context compaction — summarizing or discarding earlier parts of the conversation once the window starts filling up, while preserving the decisions and facts that still matter — is the harness's answer. Done well, it's invisible: the agent keeps working, its plan and its understanding of what's been tried survive, and only the verbose intermediate noise (a long file dump that's already been acted on, a tool call whose result has already been incorporated into a decision) gets dropped. Done badly, an agent forgets a constraint stated ten minutes ago and re-does work, or worse, re-introduces a bug it already fixed because the fix and the reasoning behind it got compacted away without a trace. I covered the closely related problem of persisting useful facts across sessions, not just within one, in a separate piece on agent memory — compaction is about surviving one long session; memory is about a system remembering something useful the next time it starts cold. They rhyme, but they solve different problems and a good harness needs both.

Why does a verification loop matter more than a confident model?

This is the mistake that started this whole article. A model asked "did that work" will answer based on its own generated output, not on ground truth — it has no privileged access to whether the file actually saved, the test actually passed, or the deployment actually succeeded, unless the harness goes and checks. Verification is the harness's job of feeding real, external signal back into the loop: run the test suite and paste in the actual output, run the linter and show the actual errors, hit the actual deployed endpoint and check the actual status code. The model reasoning over that real signal is doing something meaningfully different from a model reasoning over its own unverified narration of what it assumes happened.

What is sub-agent orchestration, and why bother?

A single agent's context window is a shared, depleting resource — every tool call, every file read, every intermediate exploration step eats into the same budget the final answer has to fit inside. Sub-agent orchestration is the harness pattern of spawning a separate agent instance with its own fresh context window to handle a self-contained sub-task — search the codebase for every usage of a function, summarize a long document, run an isolated investigation — and returning only the distilled result to the parent's context, instead of the parent doing that work inline and burning its own budget on intermediate noise it doesn't need to keep.

The design decision that actually matters here is what crosses the boundary back to the parent: a sub-agent that dumps its entire raw transcript back defeats the purpose, since the parent's context fills up anyway just one hop removed. A well-designed harness treats a sub-agent's return value as a report, not a transcript — a few sentences of synthesis and the concrete findings, not the process that produced them. This is also where the permission model gets subtle: a sub-agent spawned to do read-only research shouldn't inherit write access it doesn't need, and a harness that doesn't scope permissions per sub-agent is one prompt-injected search result away from a sub-task doing something the parent task never asked for.

# conceptual shape of a sub-agent call from a harness's perspective
def spawn_subagent(task_description, allowed_tools, isolation="fresh_context"):
    subagent = Agent(
        context=fresh_context(),
        tools=scope_tools(allowed_tools),   # never inherit the parent's full tool set by default
        system_prompt=task_description,
    )
    result = subagent.run_to_completion()
    return summarize(result)  # a report, not the raw transcript

What should you actually build versus adopt?

Most teams building an "agent" in 2026 are not building a novel harness from scratch — they're composing one from an existing agent framework or CLI, wiring in tools via something like the Model Context Protocol so tool integrations are portable across harnesses rather than hand-rolled per project, and spending their actual engineering effort on the pieces specific to their domain: what tools exist, what the permission policy should be for their environment, and what "verified" means for their specific task. That's the right allocation of effort. The agentic loop itself, context compaction, and the low-level plumbing of tool-call parsing are solved problems with mature, well-tested implementations; reinventing them from scratch is exactly the kind of infrastructure work that doesn't differentiate anything and is easy to get subtly wrong in ways that only show up under load — a compaction bug that drops a safety instruction, a permission check that has a gap for one tool type nobody tested.

Where teams should spend custom effort is the parts that are genuinely specific to them: the verification step (what does "actually done" mean for a database migration versus a customer support reply versus a robot fleet policy update — three domains this blog has covered from three different angles), the permission policy (what's genuinely safe to auto-approve in your environment versus what needs a human in the loop), and the tool set itself (narrow, well-described, well-erroring tools beat one giant do-anything shell tool almost every time). For regulated environments specifically, the permission and audit-trail questions get sharper still — I go deeper on that in a separate piece on multi-agent design under regulatory constraints.

What to carry away

An agent's competence is a product of the model's reasoning and the harness's discipline, and in practice the harness accounts for more of the variance between a reliable agent and an unreliable one than most teams expect going in. The loop, the tool schema, the permission gate, context compaction, the verification step, and sub-agent scoping are all harness decisions, not model decisions — and every one of them is a place a well-engineered agent and a fragile one diverge, independent of which model sits at the center. If you're evaluating whether to build a custom harness or adopt an existing agent framework, the honest default in 2026 is to adopt the loop and spend your engineering budget on the verification logic and permission policy specific to your domain — that's where the actual risk and the actual differentiation both live.