Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity¶
🕒 Published (v1): 2026-07-15 10:26 UTC · Source: Arxiv · link
Why this paper was selected
Self-evolving harness via quality-diversity; directly buildable; rare harness-first framing
Ask a follow-up
Open an assistant pre-loaded with this paper's context.
💬 Ask ChatGPT✦ Ask Claude
TL;DR¶
This paper proposes a self-evolving framework for LLM agent harnesses that cleanly separates proposing patches (done by a stronger LLM evolver) from crediting them (done by deterministic code with paired 2σ significance testing and a sealed held-out test). A Gated Semantic MAP-Elites (GSME) archive keyed on diagnosed failure pathologies rather than task identity provides an anti-overfitting inductive bias. Across seven domains with a frozen 27B model, credited sealed-test gains of +9 to +15.5 pp retain 86–147% of training lift.
Problem¶
Prior harness auto-evolution systems credit improvements via peak pass rate over many rounds on a single task set with no held-out split. This conflates genuine improvement with measurement noise (single-run pass@1 swings ≈5 pp on borderline tasks), lucky draws, or overfitting to specific training tasks. The result is gains that are neither auditable nor reproducible. Separately, greedy search over edits collapses onto safe prompt tweaks once bolder changes fail, missing structural harness improvements.
Method¶
The system pairs a frozen task agent \(M\) with a separate evolver \(D\) (Claude Opus 4.8 via Claude Code, \(D \neq M\)). \(D\) reads failing trajectories, assigns a two-axis (where × why) pathology (where ∈ {prompt, knowledge, runtime, config}; why ∈ a small evolvable set of failure modes such as "thinking-runaway," "premature finalization"), and proposes env-gated patches. All quantitative steps are owned by deterministic code.
Three gates filter candidates in increasing cost order: 1. Validity gate — pre-/post-scoring environment health check; infra failures trigger re-runs (up to 2×) rather than penalizing the agent. 2. Activation gate — each patch must have fired (via a declared activation beacon) for its effect to be attributed. 3. Significance gate — a per-task paired 2σ test (\(z \geq 1.96\)) on the full set at \(K=3\); per-task pairing removes between-task difficulty variance.
Credited elites populate GSME: a categorical archive with one elite per (where × why) cell. Selection is quality-biased (extend the global best), but cross-cell recombination stacks elites from multiple pathology cells, exploiting diversity without the cost of uniform illumination. The archive substrate is a git tree (each candidate is a branch + commit) for auditability. A cold-start baseline is scored on the full train set at \(K=3\); the sealed test is scored exactly once after evolution terminates, never during.
A cost-reduction theorem (Theorem 1) shows tree-aware score borrowing reproduces all 14 credited decisions at 67% fewer child trials (validated offline 14/14 and live 7/7).
Key Contributions¶
- Auditable propose-vs-credit loop: LLM owns semantics; deterministic code owns sampling, gates, and significance; every credited gain is reproducible.
- Gated Semantic MAP-Elites (GSME): pathology-keyed categorical archive with cross-cell recombination; accepted edits span all four harness levers (not only prompts); on 4/6 domains the selected harness is a cross-cell recombination.
- Held-out generalization evidence: sealed-test retention 86–147% across six credited benchmarks, a positive result where prior work documents overfitting (VeRO, AIRA_2).
- Pathology-to-patch matching law: the winning patch tracks the model's dominant pathology, not its size or family; the same pathology (careless→submit-verify) replicates across Qwen and Gemini families on AppWorld.
- Tree-aware score borrowing recipe: 67% cost reduction for growing benchmarks, validated exhaustively.
Results¶
- Six of seven domains credited at \(z \geq 1.96\) (p ranging \(< 10^{-4}\) to \(0.033\)):
| Benchmark | train Δ (pp) | sealed Δ (pp) | z | retention |
|---|---|---|---|---|
| TB2 | +6.3 | +9.3 | 2.71 | 147% |
| LiveCode | +15.8 | +13.7 | 2.65 | 86% |
| Omni-MATH | +10.1 | +11.7 | 3.66 | 115% |
| BrowseComp+ | +15.8 | +13.9 | 3.96 | 88% |
| GDPval | +8.4 | +9.2 | 2.13 | 109% |
| AppWorld | +17.8 | +15.5 | 6.44 | 87% |
| SWE-bench (prelim.) | +13.9 | +5.1 | 0.78 | 37% |
- pass@3 also rises +5.6–15.4 pp on all credited domains, confirming expanded solvable-task set rather than variance reduction.
- Cross-model study (AppWorld): evolving Qwen3.5-397B-A17B and Gemini 3 Flash shows model-family-independent replication of the careless→submit-verify pathology-patch pair.
- Ablation (LiveCode train): selective reasoning-runaway recovery beats a 16× larger token budget and a blanket thinking-off toggle.
- The significance gate is two-sided: on TB2 it flagged and culled a significantly negative candidate (−6.4 pp).
- AHE baseline (closest prior): terminal-bench-2 69.7→77.0; this work achieves 77.3 sealed (vanilla baseline differs, direct comparison approximate).
Limitations¶
- Mechanical enforcement of test-set withholding (preventing accidental consultation) is explicitly deferred to future work.
- SWE-bench held-out set is only \(n=26\); a real +5 pp effect sits below the paired-2σ bar (\(z=0.78\)), leaving that domain uncredited.
- AppWorld has high run-to-run variance (vanilla self-rerun drifted ∼9 pp); smaller candidate effects were unresolvable there.
- The evolver requires a stronger external model (\(D \neq M\), here Claude Opus 4.8), adding inference cost and a dependency on a proprietary API.
- Failure-mode diagnosis accuracy is limited: reflection studies cited by the authors find end-to-end root-cause diagnosis below 30% even for strong models; the framework survives this via the propose-but-never-credit split, but diagnosis quality caps the proposal quality.
- Evaluated only on frozen weights; weight-updatable regimes are out of scope.
- The why label is LLM-assigned and potentially noisy; the archive degrades gracefully but label noise is not formally characterized.
Relevance to Agentic AI / LLM Agents¶
This work directly tackles a central deployment reality: for production agents with frozen weights, the harness—prompt, knowledge injection, control loop, configuration—is the only improvable lever, yet systematic, trustworthy harness optimization has lacked rigorous credit assignment. The propose-vs-credit split and paired significance gating establish a measurement discipline that prior harness-evolution work (Self-Harness, HarnessFix, AHE, Meta-Harness) uniformly lacks, making the gains auditable rather than inflated. The GSME pathology-keyed diversity archive is directly relevant to multi-domain agent deployment: it resists the known failure mode of greedy search collapsing to prompt edits and produces harnesses that exercise runtime and config levers—precisely the components that most differentiate capable long-horizon agents. The pathology-to-patch matching law reframes harness engineering as model-specific failure-mode correction, which has practical implications for teams maintaining agents across rapidly turning-over model releases.