Skip to content

Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization

🕒 Published (v1): 2026-04-22 16:00 UTC · Source: Arxiv · link

Why this paper was selected

Textual parameter graph optimization enables self-improving multi-agent system design

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TPGO reframes multi-agent system (MAS) optimization as a graph evolution problem by representing agents, tools, and workflows as a Textual Parameter Graph (TPG) and applying natural-language "textual gradients" derived from execution traces. Its core meta-learning module, GRAO, builds a persistent memory of past optimization attempts and uses retrieval-augmented in-context learning to make the optimizer itself improve over iterations. On GAIA and MCP-Universe, TPGO achieves +10.6% and +7.86 pp absolute success-rate gains over strong baselines while cutting GAIA execution time by 56%.

Problem

Existing Automatic Prompt Optimization (APO) methods (TextGrad, PromptAgent, EvoPrompt) treat MAS configuration as flat, independent prompts and cannot reason about structural interdependencies—tool descriptions, inter-agent communication protocols, workflow logic—that are frequent root causes of MAS failures. Worse, all prior optimizers are static: they apply updates for a given problem but do not internalize past optimization experiences to improve future proposals. This creates a ceiling on automated MAS improvement.

Method

TPGO operates in a closed loop over three phases:

  1. Graph Construction (TPG): A Parser LLM decomposes monolithic agent prompts into a directed graph \(G = (V, E)\) with three node types—Role (persona/objectives), Logic (reasoning protocols/constraints), Tool (API specs/examples)—and typed edges encoding intra-agent flow, tool integration, and inter-agent communication. The full configuration is \(\Theta = \bigcup_{v_i \in V} \text{Content}(v_i)\).

  2. Gradient-Driven Evolution: After running a task batch, a diagnostic LLM produces textual gradients \(\nabla_{\text{text}} = \{\delta^+, \delta^-\}\) per trajectory—positive gradients distill successful reasoning patterns, negative gradients diagnose failures. Negative gradients are embedded and clustered with DBSCAN to identify systemic error patterns. Per cluster, an Optimizer LLM generates a graph-edit proposal \(\Delta G\) (operations: REWRITE_NODE, PRUNE_EDGE, ADD_NODE, ADD_EDGE) that produces \(G' \leftarrow G \oplus \Delta G\).

  3. GRAO (Group Relative Agent Optimization): Each \(\Delta G\) is validated on the task subset that triggered its error cluster, yielding an effectiveness score \(E(\Delta G) \in [0,1]\). The triple \((C, \Delta G, E)\)—problem context, proposal, outcome—is stored in a persistent Optimization Experience Memory. For a new error cluster \(C_{\text{new}}\), GRAO retrieves semantically similar historical contexts, ranks them by \(E\), and injects the top-ranked examples as few-shot context for the Optimizer LLM, turning it into a retrieval-augmented meta-learner.

Key Contributions

  • TPG representation: first formalization of MAS configuration as a modular directed graph enabling both node-content refinement and structural topology edits.
  • Textual gradients with clustering: semantic DBSCAN over failure descriptions to target systemic root causes rather than individual trajectory noise.
  • GRAO meta-optimizer: persistent experience memory + group-relative retrieval that makes the optimizer progressively more effective, preventing catastrophic forgetting observed in static optimizers.
  • Dual optimization modes: validated on both exploratory (no gold answer, MCP-Universe) and imitative (gold answer available, GAIA) settings.

Results

  • MCP-Universe (exploratory, ReAct + GPT-4.1): success rate 30.96% → 38.82% (+7.86 pp, +25.4% relative) over 5 iterations; Web Searching subdomain 5.45% → 14.55% (+9.10 pp).
  • MCP-Universe (exploratory, ReAct + DeepSeek-V3.2): 34.77% → 41.86% (+7.09 pp); Web Searching +14.37 pp.
  • GAIA (imitative, MiroFlow + GPT-5): overall pass@1 73.8% → 81.6% (+10.6% relative); Level 3 (hardest) 44.4% → 63.6% (+43.2% relative); average execution time 4014 s → 1765 s (−56.0%).
  • Ablation (MCP-Universe Browser Automation): removing TPG drops 28.03% → 25.13%; removing structural graph edits → 24.90%; removing clustering → 26.52%; random clustering → 19.97%.
  • GRAO stability: without GRAO, iterative optimization shows catastrophic forgetting (success rate degrades after early iterations); with GRAO, performance improves monotonically across 5 iterations.

Limitations

  • Requires multiple full MAS execution passes per optimization iteration, making compute cost significant for large task batches.
  • GRAO's experience memory provides no benefit in zero-shot or first-iteration settings; gains accumulate only over multiple runs.
  • The Parser LLM's quality in decomposing monolithic prompts is a hard upstream dependency—poor decomposition corrupts the entire graph structure.
  • Evaluation uses only two agent frameworks (ReAct, MiroFlow) and two benchmarks; generalization to other MAS topologies (e.g., hierarchical, debate-based) is unverified.
  • Graph-edit operations are a fixed vocabulary; emergent structural patterns requiring novel edit types cannot be expressed.

Relevance to Harnesses / Meta-Harnesses

TPGO is directly a meta-harness: it wraps an existing MAS harness (ReAct, MiroFlow) and evolves its entire configuration—prompts, tool specs, agent wiring—without human intervention, which is the defining property of a self-improving orchestration layer. GRAO's experience memory instantiates a "harness that learns how to tune harnesses," a step beyond static prompt-tuning pipelines that only execute optimization once per run. The graph-edit vocabulary (REWRITE_NODE, PRUNE_EDGE, ADD_NODE, ADD_EDGE) provides a concrete API for programmatic harness mutation, directly relevant to anyone building automated agent-engineering scaffolds. The dual exploratory/imitative evaluation also offers a reusable protocol for benchmarking meta-harness learning under different feedback regimes.