Skip to content

From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents

🕒 Published (v1): 2026-07-08 12:09 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EvoSOP is a model-agnostic framework that converts an LLM agent's flat set of atomic actions into a self-evolving hierarchical toolset by iteratively synthesizing, merging, evaluating, and pruning Standard Operating Procedures (SOPs). Unlike one-shot tool-creation approaches, EvoSOP closes a continuous optimization loop that mirrors the machine-learning pipeline without any parametric updates. Experiments on ACEBench and Tau2Bench show consistent gains in task success rate and reductions in reasoning rounds over atomic-action and one-shot-SOP baselines.

Problem

Existing LLM-agent frameworks rely on static, flat toolsets of granular atomic actions (file I/O, single-turn search, etc.), forcing agents to re-derive low-level multi-step logic on every task. Prior work on dynamic tool creation (e.g., ASI) treats tool addition as a one-time event with no lifecycle management, leading to bloated toolsets filled with redundant, brittle, or semantically misaligned tools that increase reasoning overhead and cascading-error rates, especially in long-horizon tasks.

Method

EvoSOP runs a four-module iterative loop over mini-batched execution trajectories \(\xi\), with no gradient updates to the underlying LLM:

  1. Constructor — identifies recurring, causally coupled sequences of atomic tool calls in trajectories and rewrites them into callable SOPs with full schema \(\langle\text{name, desc, params, returns}\rangle\) plus lightweight conditional/error-handling logic: $\(\xi_i' = f_{\text{extract}}(\xi_i), \quad S_i = f_{\text{rewrite}}(\xi_i', F_{\text{atomic}})\)$

  2. Merger — within each training batch, detects functional overlaps among newly constructed SOPs and consolidates them into a single higher-order SOP; original constituent SOPs are retained (non-destructive) until empirically evaluated: $\(S_b = \bigcup_{i \in b} S_i, \quad S' = f_{\text{merge}}(S_b)\)$

  3. Evaluator — expands the toolset \(F' = F_{\text{atomic}} \cup \{f_{\text{schema}}(s) \mid s \in S_b \cup S'\}\) and re-executes all training tasks in the live environment, collecting new rollout trajectories \(\hat{\xi}\) as empirical performance data.

  4. Reviewer — categorizes each SOP invocation across \(\hat{\xi}\) as optimal, partial utility, neutral, negative interference, or implementation defect; aggregates statistics and prunes low-quality SOPs: $\(\mathcal{R} = \bigoplus_{i \in I_{\text{train}}} f_{\text{review}}(\hat{\xi}_i), \quad S_{\text{solid}} = \{s \mid f_{\text{check}}(s, \mathcal{R}_s) \neq \text{remove}\}\)$

A checkpointing mechanism selects the iteration with the highest training success rate as the final epoch, providing a decaying-learning-rate analogue as the toolset stabilizes.

Key Contributions

  • Formalizes agent self-evolution as non-parametric structural optimization, drawing an explicit analogy to the ML pipeline (data acquisition → forward propagation → symbolic backpropagation via SOP extraction → regularization via merging/pruning).
  • Proposes EvoSOP with a four-module construct–merge–evaluate–prune lifecycle for continuous toolset management, in contrast to one-shot tool-creation methods.
  • Demonstrates that the Reviewer (pruning) module is the most critical component: ablating it causes the largest performance drop, confirming that active tool management is as important as tool creation.
  • Shows model-agnosticity: SOP construction tested with GPT-4o, Gemini-3-Flash-Preview, and Qwen-Max; evaluation backbone is GPT-4o throughout.

Results

  • ACEBench Multi-Step: EvoSOP on ReAct backbone improves success rate from 45.8% (ASI) to 83.3% with GPT-4o construction; DFSDT + EvoSOP reaches 85.8% vs. 75.0% for DRAFT.
  • ACEBench Multi-Turn: ReAct + EvoSOP reaches 79.4% vs. 57.2% (ASI) with Gemini-3-FP backbone; DFSDT + EvoSOP achieves 83.3% vs. 71.1% (DRAFT).
  • Tau2Bench-Telecom (solo): ReAct + EvoSOP reaches 43.3% vs. 35.4% baseline and 37.7% (ASI) with GPT-4o construction.
  • Reasoning rounds stabilize at a significantly lower level than baseline ReAct across epochs (Figure 2), reducing API latency and long-horizon error risk.
  • Ablation: removing the Reviewer causes the largest drop on both ACEBench subsets; removing the Merger also degrades performance, confirming synergistic design.

Limitations

  • Evaluation backbone is restricted to GPT-4o; SOP construction tests three models but end-to-end cross-backbone generalization is not fully measured.
  • Tau2Bench improvement margins are smaller than ACEBench, indicating that high-entropy, stateful environments with large tool spaces remain challenging.
  • The Reviewer's categorization relies on LLM judgment, which may itself be stochastic or miscalibrated for novel failure modes.
  • Mini-batching and full environment re-execution per iteration introduce wall-clock overhead not explicitly quantified.
  • No comparison to fine-tuning or RLHF-based agent improvement; the non-parametric claim is a design choice, not an empirically demonstrated superiority over parametric alternatives.

Relevance to Harnesses / Meta-Harnesses

EvoSOP is directly relevant as a meta-harness for toolset lifecycle management: it wraps any base agent (ReAct, DFSDT) with an outer optimization loop that observes execution traces, synthesizes higher-order tools, and prunes the tool inventory — exactly the pattern of a meta-harness that governs and improves inner-loop agent behavior without touching agent weights. The construct–merge–evaluate–prune cycle is a reusable template that could be layered atop any tool-using agent framework, making it a candidate design pattern for harness builders who want automated capability accumulation. The analogy to ML pipeline stages (forward pass, symbolic backprop, regularization) provides a principled framing for reasoning about meta-harness convergence properties. Compared to single-shot harnesses that hand-craft tool schemas, EvoSOP shows that iterative empirical feedback loops are necessary for reliable tool libraries in long-horizon agent deployments.