Recursive Harness Self-Improvement¶

🕒 Published (v1): 2026-07-17 00:21 UTC · Source: Arxiv · link

Why this paper was selected

Zaharia (Berkeley/Databricks); harnesses as data-generators that self-improve — fundamental for meta-harness builders

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Recursive Harness Self-Improvement (RHI) iteratively refines user-constructed agent harnesses—represented as prompt-level specifications of the agent loop—using pairwise preference feedback over consecutive revisions. A few RHI iterations on low-reasoning-effort agents surpass the performance of maximum-reasoning-effort baselines while cutting inference cost by up to 60%, because gains come from improved inter-agent information flow rather than longer reasoning traces.

Problem¶

Provider-built harnesses are too expensive to update continually, and existing population-based harness/prompt search methods require \(\Theta(m^2)\) agent executions and pairwise evaluations per iteration, making them impractical for users who need to specialize agents to many open-ended tasks under tight budgets. The paper addresses the gap: how to achieve lightweight, few-iteration harness optimization that still yields substantial performance gains over test-time scaling.

Method¶

RHI defines the harness as a prompt-level object with four components: roles, instructions, contracts (inter-agent information contracts), and hops (workflow control-flow). It replaces the global competitor distribution \(\mu\) with a trajectory-local objective concentrating on the immediately preceding harness \(H_x^{(i-1)}\):

\[\tilde{f}_x^{(i)}(H) = \mathbb{E}_{y \sim \mathcal{A}(H,x),\; y^- \sim \mathcal{A}(H_x^{(i-1)},x)}\left[\mathbf{1}\{\mathcal{L}_\text{eval}(y, y^-; x_\text{eval}) = y \succ y^-\}\right]\]

Each iteration costs \(\Theta(1)\): one new agent execution plus one pairwise LLM evaluation (vs. \(\Theta(m^2)\) for population methods). The harness optimizer \(\mathcal{L}_\text{harness}\) receives the current harness \(H^{(i)}\) and the accumulated preference history \(\mathcal{D}_x^{(i)} = \{\mathcal{L}_\text{eval}(y^{(k)}, y^{(k-1)}; x_\text{eval})\}_{k=1}^{i}\) and produces \(H^{(i+1)}\), without directly seeing the evaluation prompt \(x_\text{eval}\). The history acts as a momentum-semantic signal in the discrete text space. RHI prioritizes updating contracts and hops over roles/instructions.

Key Contributions¶

RHI algorithm: trajectory-local, \(\Theta(1)\)-cost harness self-improvement using pairwise self-comparison history.
Formalization: shows trajectory-local objective targets the same latent utility ordering as the ideal population objective under a standard preference model.
Empirical benchmark: 30 synthetic open-ended ML research tasks across quantitative finance, robotics, and pharmacy, evaluated via LLM-as-a-judge pairwise comparison.
Information-theoretic hypothesis: RHI implicitly maximizes mutual information between each harness component and the task while penalizing redundancy across components (\(f_\text{ext}\) increases, \(f_\text{int}\) decreases).
Cost analysis: demonstrates that output-token usage remains flat across RHI iterations while cache read/write usage and inference cost decrease, ruling out "longer reasoning" as the explanation for gains.

Results¶

On 30 tasks, high-reasoning agents (sonnet-4.6, opus-4.7, opus-4.8) with RHI-improved harnesses achieve higher pairwise win counts than all test-time scaling baselines: xhigh, max, and ultracode.
For opus-4.8, RHI reduces inference cost by up to 60% relative to the ultracode baseline.
Gains emerge within a few iterations (typically 2–3 shown in figures).
Output-token usage remains nearly constant across iterations; cache read/write usage decreases, confirming gains are not from longer outputs.
RHI also raises the performance plateau of train-time scaling (ablation), though the primary claims focus on test-time scaling.

Limitations¶

Evaluated only on synthetic open-ended ML research tasks (30 tasks); generalization to real-world heterogeneous tasks is not demonstrated.
Pairwise LLM-as-a-judge evaluation introduces noise and potential bias; results are averaged over two judges and three seeds but remain subject to evaluator reliability.
Trajectory-local objective is not an unbiased estimator of the global win-rate objective; it may converge to locally preferred but globally suboptimal harnesses.
The information-theoretic hypothesis (\(f_\text{ext}\)/\(f_\text{int}\) framing) is presented as a testable hypothesis with empirical evidence but not as a proved theorem.
RHI requires at least one full agent execution per iteration per task; for very expensive tasks, even \(\Theta(1)\) per-iteration cost may be non-trivial.
Harness representation is prompt-level (text), limiting expressiveness compared to code-level harness evolution methods.

Relevance to Agentic AI / LLM Agents¶

RHI directly addresses the harness-in-the-loop learning problem—optimizing the scaffolding that orchestrates multi-agent workflows—which is increasingly central to frontier AI system design as models and harnesses co-evolve. By framing the harness as a prompt-level specification of the agent loop (roles, instructions, contracts, hops) and showing that iterative self-refinement via pairwise preference outperforms brute-force test-time scaling, RHI offers a practical and computationally cheap path for users to specialize coding agents to open-ended tasks. The information-theoretic analysis—positing that RHI implicitly maximizes task-relevant mutual information while reducing inter-component redundancy—provides a principled lens for understanding why better inter-agent information flow (rather than deeper reasoning) drives agentic performance gains, a finding with broad implications for multi-agent system design.