Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity¶
🕒 Published (v1): 2026-07-15 10:26 UTC · Source: Arxiv · link
Ask a follow-up
Open an assistant pre-loaded with this paper's context.
💬 Ask ChatGPT✦ Ask Claude
TL;DR¶
This paper proposes a self-evolving framework for LLM agent harnesses that cleanly separates proposing patches (done by a stronger LLM evolver) from crediting them (done by deterministic code with paired 2σ significance testing and a sealed held-out test). A Gated Semantic MAP-Elites (GSME) archive keyed on diagnosed failure pathologies rather than task identity provides an anti-overfitting inductive bias. Across seven domains with a frozen 27B model, credited sealed-test gains of +9 to +15.5 pp retain 86–147% of training lift.
Problem¶
Prior harness auto-evolution systems credit improvements via peak pass rate over many rounds on a single task set with no held-out split. This conflates genuine improvement with measurement noise (single-run pass@1 swings ≈5 pp on borderline tasks), lucky draws, or overfitting to specific training tasks. The result is gains that are neither auditable nor reproducible. Separately, greedy search over edits collapses onto safe prompt tweaks once bolder changes fail, missing structural harness improvements.
Method¶
The system pairs a frozen task agent \(M\) with a separate evolver \(D\) (Claude Opus 4.8 via Claude Code, \(D \neq M\)). \(D\) reads failing trajectories, assigns a two-axis (where × why) pathology (where ∈ {prompt, knowledge, runtime, config}; why ∈ a small evolvable set of failure modes such as "thinking-runaway," "premature finalization"), and proposes env-gated patches. All quantitative steps are owned by deterministic code.
Three gates filter candidates in increasing cost order: 1. Validity gate — pre-/post-scoring environment health check; infra failures trigger re-runs (up to 2×) rather than penalizing the agent. 2. Activation gate — each patch must have fired (via a declared activation beacon) for its effect to be attributed. 3. Significance gate — a per-task paired 2σ test (\(z \geq 1.96\)) on the full set at \(K=3\); per-task pairing removes between-task difficulty variance.
Credited elites populate GSME: a categorical archive with one elite per (where × why) cell. Selection is quality-biased (extend the global best), but cross-cell recombination stacks elites from multiple pathology cells, exploiting diversity without the cost of uniform illumination. The archive substrate is a git tree (each candidate is a branch + commit) for auditability. A cold-start baseline is scored on the full train set at \(K=3\); the sealed test is scored exactly once after evolution terminates, never during.
A cost-reduction theorem (Theorem 1) shows tree-aware score borrowing reproduces all 14 credited decisions at 67% fewer child trials (validated offline 14/14 and live 7/7).
Key Contributions¶
- Auditable propose-vs-credit loop: LLM owns semantics; deterministic code owns sampling, gates, and significance; every credited gain is reproducible.
- Gated Semantic MAP-Elites (GSME): pathology-keyed categorical archive with cross-cell recombination; accepted edits span all four harness levers (not only prompts); on 4/6 domains the selected harness is a cross-cell recombination.
- Held-out generalization evidence: sealed-test retention 86–147% across six credited benchmarks, a positive result where prior work documents overfitting (VeRO, AIRA_2).
- Pathology-to-patch matching law: the winning patch tracks the model's dominant pathology, not its size or family; the same pathology (careless→submit-verify) replicates across Qwen and Gemini families on AppWorld.
- Tree-aware score borrowing recipe: 67% cost reduction for growing benchmarks, validated exhaustively.
Results¶
- Six of seven domains credited at \(z \geq 1.96\) (p ranging \(< 10^{-4}\) to \(0.033\)):
| Benchmark | train Δ (pp) | sealed Δ (pp) | z | retention |
|---|---|---|---|---|
| TB2 | +6.3 | +9.3 | 2.71 | 147% |
| LiveCode | +15.8 | +13.7 | 2.65 | 86% |
| Omni-MATH | +10.1 | +11.7 | 3.66 | 115% |
| BrowseComp+ | +15.8 | +13.9 | 3.96 | 88% |
| GDPval | +8.4 | +9.2 | 2.13 | 109% |
| AppWorld | +17.8 | +15.5 | 6.44 | 87% |
| SWE-bench (prelim.) | +13.9 | +5.1 | 0.78 | 37% |
- pass@3 also rises +5.6–15.4 pp on all credited domains, confirming expanded solvable-task set rather than variance reduction.
- Cross-model study (AppWorld): evolving Qwen3.5-397B-A17B and Gemini 3 Flash shows model-family-independent replication of the careless→submit-verify pathology-patch pair.
- Ablation (LiveCode train): selective reasoning-runaway recovery beats a 16× larger token budget and a blanket thinking-off toggle.
- The significance gate is two-sided: on TB2 it flagged and culled a significantly negative candidate (−6.4 pp).
- AHE baseline (closest prior): terminal-bench-2 69.7→77.0; this work achieves 77.3 sealed (vanilla baseline differs, direct comparison approximate).
Limitations¶
- Mechanical enforcement of test-set withholding (preventing accidental consultation) is explicitly deferred to future work.
- SWE-bench held-out set is only \(n=26\); a real +5 pp effect sits below the paired-2σ bar (\(z=0.78\)), leaving that domain uncredited.
- AppWorld has high run-to-run variance (vanilla self-rerun drifted ∼9 pp); smaller candidate effects were unresolvable there.
- The evolver requires a stronger external model (\(D \neq M\), here Claude Opus 4.8), adding inference cost and a dependency on a proprietary API.
- Failure-mode diagnosis accuracy is limited: reflection studies cited by the authors find end-to-end root-cause diagnosis below 30% even for strong models; the framework survives this via the propose-but-never-credit split, but diagnosis quality caps the proposal quality.
- Evaluated only on frozen weights; weight-updatable regimes are out of scope.
- The why label is LLM-assigned and potentially noisy; the archive degrades gracefully but label noise is not formally characterized.
Relevance to Harnesses / Meta-Harnesses¶
This paper directly addresses the core meta-harness problem: how to automatically discover and validate improvements to agent scaffolding without overfitting to the training distribution. The proposal/credit separation is the key architectural insight — letting a stronger LLM generate patches freely while keeping evaluation fully deterministic and statistically gated prevents the usual failure mode where self-improvement systems hill-climb on proxy metrics that don't generalize. Keying the MAP-Elites archive on diagnosed failure pathologies rather than task identities is a practically deployable anti-overfitting inductive bias: it forces diversity across what breaks, not just what was tested, which is exactly the pressure missing from most evolutionary scaffold search. The 86–147% held-out retention result is the credibility anchor — it demonstrates that automated harness evolution can produce durable gains with a frozen base model, which is the regime most practitioners actually operate in.