From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents

🕒 Published (v1): 2026-07-08 12:09 UTC · Source: Arxiv · link

Why this paper was selected

Alibaba DAMO group; converts atomic tool calls to SOPs; buildable self-evolving system

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EvoSOP is a model-agnostic, parameter-free framework that enables LLM agents to self-evolve by synthesizing atomic tool calls into reusable Standard Operating Procedures (SOPs). It iteratively manages the toolset through a four-stage lifecycle—construction, merging, evaluation, and pruning—preventing toolset bloat and cascading reasoning errors. Experiments on ACEBench and Tau2Bench show consistent gains in task success rates and reductions in interaction rounds over static and one-shot tool-expansion baselines.

Problem

Existing LLM agent frameworks rely on static, fine-grained atomic toolsets (e.g., single file I/O, single-turn search). This forces agents to re-derive low-level multi-step logic for every recurring workflow, causing (i) excessive reasoning overhead, (ii) cascading errors in long-horizon tasks, and (iii) inability to reuse learned procedural knowledge. Prior dynamic tool-generation work (e.g., ASI) treats tool creation as a one-time event, leading to bloated toolsets with redundant, brittle, or poorly described tools that degrade decision-making.

Method

EvoSOP operates as a non-parametric, iterative optimization loop over the agent's toolset \(F\), drawing a structural analogy to the ML training pipeline:

1. Constructor (data acquisition + forward pass): Given execution trajectories \(\xi_i\), an LLM identifies frequently co-occurring, causally coupled atomic action segments and rewrites them as callable SOPs with executable code and semantic schemas: \(\xi_i' = f_\text{extract}(\xi_i)\), \(S_i = f_\text{rewrite}(\xi_i', F_\text{atomic})\).

2. Merger (structural regularization): Inspects the batch-union \(S_b = \bigcup_{i \in b} S_i\) for functional overlap, merging redundant SOPs into generalized higher-order tools \(S' = f_\text{merge}(S_b)\) without immediately deleting constituents (non-destructive consolidation).

3. Evaluator (empirical validation): Loads \(F' = F_\text{atomic} \cup \{f_\text{schema}(s) \mid s \in S_b \cup S'\}\), re-executes all training tasks in the real environment, and captures new trajectories \(\hat{\xi}\) to observe actual tool utility and reliability.

4. Reviewer (backward propagation + pruning): Categorizes each SOP invocation as Optimal, Partial Utility, Neutral, Negative Interference, or Implementation Defect. Aggregates statuses across the training batch and prunes underperforming SOPs: \(S_\text{solid} = \{s \mid f_\text{check}(s, R_s) \neq \text{remove}\}\).

Training uses mini-batching and checkpointing—the iteration with highest training success rate is selected as the final epoch.

Key Contributions

• Formalizes a self-evolution paradigm for agents as non-parametric structural optimization, drawing explicit parallels to ML pipeline stages (forward pass, backward propagation, regularization, learning-rate decay).
• Proposes EvoSOP, a four-module framework (Constructor, Merger, Evaluator, Reviewer) for continuous SOP lifecycle management rather than one-shot tool addition.
• Demonstrates that active tool management (pruning/merging) is as critical as tool creation, explaining the failure mode of one-shot SOP approaches.
• Framework is model-agnostic and black-box compatible—no gradient updates or fine-tuning required.

Results

• ACEBench Multi-Step: EvoSOP on ReAct achieves 83.3–85.8% success (backbone-dependent) vs. 45.8–80.8% for ReAct+ASI and 60.0–80.8% for vanilla ReAct (GPT-4o backbone); gains of 2.5–13.4% over baselines including DFSDT+DRAFT.
• ACEBench Multi-Turn: EvoSOP reaches 79.4–84.4% vs. 46.7–72.2% for ReAct+ASI; consistent improvements across GPT-4o, Gemini-3-Flash-Preview, and Qwen-Max toolset construction backends.
• Tau2Bench-Telecom (solo): EvoSOP achieves 40.2–43.3% vs. 35.4–39.5% for ReAct and 37.1–37.7% for ReAct+ASI; smaller but consistent gains in a high-entropy, large-tool-space domain.
• Interaction rounds: Averaged reasoning rounds stabilize at significantly lower levels than baselines across epochs (Figure 2), reducing API cost and long-context failures.
• Ablation: Removing Reviewer causes the largest performance drop; removing Merger also hurts, confirming both contribute independently.

Limitations

• Toolset construction relies on LLM-generated code, which may introduce subtle logic defects (e.g., off-by-one in ordered message deletion) that require multiple evaluation cycles to surface.
• The Evaluator requires re-executing all training tasks in a real environment each iteration—computationally expensive at scale.
• Checkpointing selects the best training-success epoch, with no held-out validation set; this may not generalize if training tasks are not representative.
• Experiments use GPT-4o as the evaluation backbone throughout; generalization to weaker or open-source inference models is not demonstrated.
• The paper's Tau2Bench results show smaller margins, suggesting gains compress in high-complexity, large-state-space domains.

Relevance to Agentic AI / LLM Agents

EvoSOP directly addresses the hierarchical abstraction gap in tool-augmented agents—the inability to accumulate reusable procedural knowledge across tasks—which is a core bottleneck for long-horizon agentic competence. The formalization of non-parametric self-evolution as structural toolset optimization (vs. weight updates) offers a compelling complementary axis to RLHF/SFT-based agent improvement, and the Reviewer-driven pruning mechanism is a practical answer to toolset bloat, a failure mode that will worsen as agents operate in richer environments. This work connects directly to the broader line of research on skill/tool libraries for agents (e.g., Voyager, ASI) but advances it by closing the lifecycle loop—creation, consolidation, validation, and pruning—making it relevant to anyone building production agentic systems that must remain reliable over extended deployment.