The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI¶

🕒 Published (v1): 2026-07-08 01:58 UTC · Source: Arxiv · link

Why this paper was selected

Empirically frames how orchestration design drives token cost; directly actionable for harness builders

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

The paper argues that the harness—the orchestration layer wrapping model calls—controls more of an enterprise agent's token bill than model choice does, and demonstrates this via a controlled swap of two orchestration layers across the same 22 tasks and six foundation models. Replacing a conventional production agent loop with the Writer Agent Harness cuts blended cost per task by 41%, median wall-clock by 44%, and tokens per task by 38%, with quality holding at parity. A formal framework, six mechanism families, and the concept of harness leverage explain and quantify the effect.

Problem¶

Enterprise agentic systems exhibit token maxing: quality is purchased by monotonically growing token intensity—longer reasoning traces, full-history replay that grows $O(k^2)$ in turn count, broadcast tool schemas, unconstrained retrieval—at declining marginal quality per token. Falling per-token prices mask the problem via a Jevons dynamic: unit cost drops but total spend rises. Nearly all published efficiency work either targets single-call prompt compression or inter-model routing, accepting the orchestration layer as fixed, despite that layer controlling every input-side term of the token bill except the model's own verbosity.

Method¶

A natural experiment made possible by the layered architecture: hold tasks and models constant, swap only the orchestration layer.

Task set: 22 locked, capability-audited enterprise tasks (not disclosed in detail), with a pinned public price table applied uniformly to trace-recorded token counts.
Models: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6 (five vendors, three weight classes).
Baseline: The conventional production agent loop frozen on 2026-06-07: ~49 KB monolithic system prompt replayed every turn, regex-parsed XML tool calls, destructive middle-truncation on overflow, polling-based waits, no sub-agent delegation.
Treatment: Writer Agent Harness at general-availability configuration: model-agnostic execution path, native tool calling only, structured non-destructive compaction, durably journaled state, zero-token waits, sub-agent delegation.
Measurement: Trace-level token and wall-clock accounting per turn; LLM-as-judge quality score $Q \in [0,1]$; efficiency metrics $\eta_\$ = Q/C$ and $\mathrm{CPM} = Q \cdot 10^6 / \tau$.

Token economics framework: Per-task cost over a $k$-turn loop: $$C = \sum_{i=1}^{k}\!\left(p_{\mathrm{in}} T_i^{\mathrm{in}} + p_{\mathrm{out}} T_i^{\mathrm{out}}\right), \quad T_i^{\mathrm{in}} = S_i + H_i + G_i + R_i + U_i$$ Naive full-history replay gives $\sum T_i^{\mathrm{in}} \approx kS + \frac{k(k-1)}{2}\bar{m} + k\bar{G} + \sum_i R_i$ ($O(k^2)$). The effective input price under prefix caching with cache-hit fraction $h$ and discount multiplier $\kappa \approx 0.1$: $$p_{\mathrm{in}}^{\mathrm{eff}} = p_{\mathrm{in}}\!\left[1 - h(1-\kappa)\right]$$

Six harness mechanism families: 1. Cache-shape discipline: two-zone prompt—byte-stable prefix (tool schemas, stable system prompt, append-only transcript) holding up to four provider cache breakpoints; volatile tail (clock, file listings, plan state) structurally excluded from the prefix. 2. Structured, incremental, cache-aware compaction: at 80% of input budget, older history is folded into a typed checkpoint via a cheaper helper model; live tail of 4–12 most recent messages always survives verbatim; converts $O(k^2)$ to $O(k)$. 3. Context offload: sub-agents act as context firewalls returning ≤8 KB summaries; progressive skill disclosure; bulky tool outputs spill to workspace files (beyond 20 K characters); plan state is event-sourced and rendered compactly. 4. Zero-token waiting: runs suspend durably on a write-ahead log and resume on ingress events, with no polling turns; crashed runs resume from journaled state rather than re-running from scratch. 5. Failure-spend governance: typed failure classification; mid-stream failures become discarded attempts (no durable side effects); circuit breaker on byte-identical failing tool calls (3 strikes); loop capped at 50 iterations, tool parallelism at 4. 6. Model-agnostic floor: provider streams normalized to one chunk contract; schema hygiene for weaker models ($\$\mathrm{refs}$ inlined, double-encoded JSON recovered, overloaded schemas split); route plan supplied as data, no branching on model name.

Key Contributions¶

Formal token-economics decomposition at the orchestration layer: per-task spend as a function of harness-controlled terms (Eqs. 1–4), definition of token maxing as a measurable development trajectory, and the effective input price model under prompt caching.
Controlled harness-swap methodology with locked task set, frozen baseline, and trace-level accounting, isolating orchestration effects from model effects across six models.
Empirical finding of harness leverage: the quality gain a model extracts from the harness correlates almost perfectly ($r = 0.99$, $n = 6$) with its baseline strength—the harness raises the floor; the model sets the ceiling.
Mechanism inventory of six families mapped onto the cost decomposition, plus a documentation-based architectural comparison of six widely used agent systems (Claude Code, Claude Cowork, LangGraph, CrewAI, AutoGen/AG2, Hermes Agent) on the same axes.
Economic argument for harness efficiency compounding: a harness improvement multiplies every model's cost by $(1 - s_m)$ simultaneously ($s_m \in [0.33, 0.61]$), making it vendor-race-invariant.

Results¶

Blended cost per task: $0.21 → $0.12 (−41%)
Median wall-clock: 48 s → 27 s (−44%, 1.8× faster)
Tokens per task: 14.2 k → 8.8 k (−38%)
Task-completion quality: 0.78 → 0.81 (directional only; $n = 22$)
Per-model cost reduction: −33% (Gemini 3.1) to −61% (Flash 3.5); every model cheaper
Quality per dollar ($\eta_\$$): +82%
Task-completions per million tokens (CPM): 54.9 → 92.0 (+68%)
Harness vs. model choice: switching from most to least expensive model under the baseline saves 36%; adopting the harness with any model saves 33%–61%
Cache hit rate (measured on identical-prefix call): 7,876 of 7,886 prompt tokens served as cache reads (99.9%)
Sub-agent delegation (new capability, absent in baseline): reliable only on the two strongest models (quality 0.85–0.86)
Quality regressions: 7 of 48 capability×model cells regress, all on three smaller models, concentrated in orchestration-heavy capabilities (MCP tool use, multi-step Playbooks)
Harness leverage: $r = 0.99$ correlation between model baseline strength and quality gain from harness

Limitations¶

Small task set ($n = 22$); the quality improvement 0.78 → 0.81 is explicitly characterized as directional and not statistically significant at this sample size.
Self-serving evaluation: Writer, Inc. evaluates its own harness; no independent replication.
Comparison of six other agent systems (Table 1) is based on documentation and source review, not head-to-head measurement.
Workload specificity: all 22 tasks are enterprise-oriented; generalizability to other task distributions (code generation, long-horizon planning, scientific reasoning) is not established.
Capability floor means advanced orchestration features (sub-agents, multi-step Playbooks) degrade smaller/weaker models rather than helping them.
Mechanism attribution is aggregate; the contribution of each of the six families individually is not isolated.

Relevance to Agentic AI / LLM Agents¶

This paper provides the first formally grounded, empirically controlled analysis of how the orchestration layer—rather than the model—dominates the token economics of multi-turn agentic execution, directly relevant to anyone designing or evaluating agent infrastructure. The harness leverage finding ($r = 0.99$) is a strong empirical signal that model–harness co-design matters: stronger models amplify good orchestration rather than being interchangeable plug-ins. The six mechanism families (cache-shape discipline, compaction, context offload, zero-token waits, failure governance, model-agnostic floor) constitute a concrete engineering taxonomy for researchers building production agent runtimes. The $O(k^2) \to O(k)$ token-growth conversion and effective-input-price model under prefix caching are directly actionable analytical tools for reasoning about multi-agent cost scaling.