Mem$^2$Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation¶

🕒 Published (v1): 2026-04-13 02:44 UTC · Source: Arxiv · Venue: ACL 2026 · link

Why this paper was selected

Co-evolutionary capability expansion and experience distillation for self-evolving agents [ACL]

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Mem²Evolve is a self-evolving agent framework that jointly evolves capabilities (via dynamic tool/expert-agent creation) and experience (via trajectory distillation), rather than treating the two processes in isolation. A dual-memory architecture — Asset Memory $M_A$ and Experience Memory $M_E$ — couples a forward inference loop (reuse-or-create) with a backward evolution loop (validate-and-distill). Across 8 benchmarks, it outperforms the best pure-capability and pure-experience baselines by 6.46% and 11.80% average Pass@1, respectively.

Problem¶

Prior self-evolving agent frameworks fall into two isolated paradigms: (1) experience-centric systems (DyLAN, DSPy, AFlow) that optimize execution over a fixed, manually-predefined toolset, capping capability growth; and (2) capability-centric systems (Alita, ToolMaker, AgentVerse) that dynamically create new tools or expert agents from scratch without experiential guidance, producing non-replicable successes and repeated errors. Neither paradigm exploits the complementary relationship between accumulated experience and expanding capability.

Method¶

Dual-Memory Mechanism. $M_A = B_{agt} \cup B_{tool}$ stores reusable expert-agent specifications $m_{agt} = \langle\rho, \epsilon, \sigma, T_{avail}\rangle$ (role, expertise, strategy hints, available tools) and MCP-compliant executable tools $m_{tool} = \langle n, d_{func}, c_{impl}, \omega_{doc}\rangle$. $M_E = E_{agt} \cup E_{tool}$ accumulates memory items $e = \langle h_{title}, d_{desc}, U_{case}, \kappa_{content}\rangle$ distilled from both successful and failed trajectories.

Forward Inference ("Reuse first, Create on demand"). For each sub-task $s_i$ the Recruitment Function $\Gamma(s_i)$ computes similarity to $M_A$: $$\Gamma(s_i) = \begin{cases} m^* & \text{if } \text{sim}(s_i, M_A) \geq \delta \\ \text{Create}(s_i \mid M_E, \text{Web}) & \text{otherwise} \end{cases}$$ When creation is triggered, tool generation conditions on retrieved tool experiences and web search results (Eq. 6). Agents execute sub-tasks via ReAct (think/action/observation).

Backward Evolution. An LLM-as-Judge produces $(r_t, c_t)$ — a binary success flag and critique. Asset persistence is gated by $\text{Valid}(m^{new}, c_t)$: the LLM synthesizes unit tests from the critique; assets failing tests enter a Self-Correction Loop before being admitted to $M_A$. Experience items are produced by a Reflection function that generalizes from successes (strategies, coordination patterns) and diagnoses failures (anti-patterns, failure–fix pairs) into $M_E$.

Key Contributions¶

First framework to propose co-evolutionary coupling of dynamic capability expansion and experience distillation for LLM agents.
Dual-memory architecture ($M_A$ + $M_E$) with a forward-backward loop enabling continuous, stable self-evolution.
Experience-guided asset creation: conditions tool/agent generation on retrieved past experience, reducing debugging iterations and increasing first-pass tool validity.
Self-Correction Loop with LLM-synthesized unit tests gates asset persistence, preventing low-quality assets from corrupting $M_A$.
Demonstrated cross-task memory reuse and single-task sustained evolution beyond prior frameworks.

Results¶

All baselines use GPT-5-chat as the backbone.

Mem²Evolve: 70.24% average Pass@1 across 8 benchmarks (GAIA L1–L3, ALFWorld, HotpotQA, 2WikiMultihopQA, AIME24/25, TravelPlanner, WebShop).
vs. best capability-centric baseline (Alita, 63.78%): +6.46%
vs. best experience-centric baseline (AFlow, ~58–59%): +11.80%
vs. GPT-5-chat ReAct (58.44%): +18.53% (naive-LLM upper bound)
On GAIA (hardest general-assistant benchmark): 88.68/82.56/57.69% at L1/L2/L3 vs. Alita's 81.13/75.58/46.15%.
Experience-guided tool creation (Table 4): first-pass validity improves by up to +56.0% relative on GAIA; average debugging iterations reduced by ~68% on AIME24/25.
Ablation: removing Tool Creation is the largest single ablation drop (−10.28%); removing Agent Memory (−4.73%) and Tool Memory (−3.13%) confirm both experience components are necessary.

Limitations¶

Backward evolution assumes ground-truth labels are inaccessible; the paper notes that when labels are available, supervision could further improve evolution — implying the current LLM-as-Judge evaluation is a weaker signal.
Self-Correction Loop and LLM-synthesized unit tests add latency and token cost per task; no wall-clock or cost analysis is reported.
Tool Bank uses MCP compliance for interoperability, but the generality across LLM backends beyond GPT-5-chat is not empirically validated.
Asset quality relies on the LLM's intrinsic code-generation and self-reflection ability; failure modes when the base model is weaker are not characterized.
Cross-task transfer experiments are described qualitatively; quantitative cross-task generalization numbers are limited to appendices.

Relevance to Harnesses / Meta-Harnesses¶

Mem²Evolve is directly relevant as a meta-harness that manages its own tool registry and agent roster at runtime — the Asset Memory is structurally a self-modifying harness that spawns, validates, and persists new sub-harness components (tools and expert agents) without human intervention. The backward-evolution loop, particularly the LLM-synthesized unit-test gating, is a harness-level quality-control mechanism analogous to CI gates in static harnesses. The Experience Memory acts as a cross-run state store that informs future harness configuration decisions, which is the key capability static harness frameworks (DyLAN, AFlow) lack. For researchers building meta-harnesses, the experience-guided creation pattern (Eq. 6) and the Self-Correction Loop offer concrete, implementable primitives for making a harness self-improving rather than statically configured.

Mem\(^2\)Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation¶