Designing a harness

Start from the job, not from the tools. A harness designed around a specific task is simpler, cheaper and easier to test than one built to do everything.

Rule of thumb: give the model only what this task needs, let it touch only what it needs to touch, and check its work in the same place every time.

Start with the task and its failures

Before you design anything, write down:

The job: one sentence. "Answer questions about our refund policy." "Draft a weekly stock report from these three sources."
What done looks like: the output or action that counts as success.
How it fails today: if there is a prototype, collect five or ten failures. These become the requirements.
What must never happen: the irreversible or expensive mistakes.

Those failures are your error analysis from the evals path. They tell you which of the five parts needs work.

Walk through the five parts

1. Context: what does the model see?

Write the standing instructions as short, specific rules. Put them in one file the system loads every time (many teams use AGENTS.md or CLAUDE.md).
Give documents as search results or on request, not all at once. A model with 200 pages in front of it is easily distracted.
Decide what carries over between steps, and what gets summarised when a long task fills the context.

2. Action: what can it touch?

Give it the fewest tools that do the job. Each extra tool is another way to go wrong.
Write each tool description as if for a new colleague: what it does, when to use it, what it returns, and what it must not be used for.
Prefer tools that return clear, structured results. A tool that returns a page of text is harder for the model to use than one that returns the three fields it needs.

3. Orchestration: how is the work split?

Decide whether one step is enough, or whether the task needs a plan. Simple jobs do not need planning.
Keep a visible state (a checklist, a file, a short log) that the agent updates. This is what stops it drifting on long tasks.
Give sub-agents one narrow job each, with its own context, and only the result comes back.
Set a limit on retries. Without one, a stuck agent can loop for a long time.

4. Control: what is it allowed to do?

Permissions: read-only by default. Writes, sends and purchases need explicit approval.
Sandbox: run code and untrusted content in an isolated environment that cannot reach your real data.
Limits: a maximum number of steps, time and spend per task.
Output checks: validate structure (valid JSON, required fields) in code before anything downstream uses the output.

5. Feedback: how will you know?

Log every step, tool call and result, so a failure can be traced.
Give the agent a way to check its own work: run the tests, re-read the source, compare against the requirement. "Done" should mean "the check passed", not "I think it's done".
Connect the failures from your list to evals (see writing rubrics), so you can measure the next version.

A worked example

Suppose you want an agent to answer staff questions about a company's expenses policy.

Job: answer policy questions, quoting the policy section.
Failures seen in a prototype: invents limits (3 times in 20 questions); answers from memory of an old policy.
Context: one instruction file saying "only answer from the policy; quote the section; say so if the policy does not cover it". Policy searched on request, current version only.
Action: one tool, search_policy(query), returning the matching section with its date. No write tools.
Orchestration: one step, no planning. No sub-agents.
Control: read-only; a fixed answer ("I can't find that in the policy") if the search returns nothing.
Feedback: every question and retrieved section logged. The invented-limit failure becomes a binary eval, run before each change.

Notice that the fix for the invented limits was mostly in the harness (the instruction and the search tool), not in changing the model.

Previous← Harness basics NextHarness engineering →

Sources

The worked example is illustrative, not from a real project. The checklist is Human Agency's synthesis.