Skip to content

SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SE-Agent is a self-evolution framework for LLM-based agents that iteratively optimizes multi-step reasoning trajectories via three operations—revision, recombination, and refinement—rather than treating each trajectory independently. Applied to SWE-bench Verified (real-world GitHub issue resolution), it achieves state-of-the-art open-source performance of 80.0% resolution rate with Claude-4-Sonnet and 61.2% with Claude-3.7-Sonnet.

Problem

Existing multi-step agent frameworks (ReAct, MCTS-based SWE-Search) treat trajectories as independent entities, missing cross-trajectory synergies. Even with diverse sampling strategies, agents converge on structurally similar paths due to the high-probability bias of autoregressive LMs, leading to homogeneous outcomes and inability to escape local optima in the solution search space.

Method

SE-Agent maintains a pool of candidate trajectories and evolves them across iterations through three operations:

  1. Revision: Generates an initial pool via multi-planning exploration (varied prompts/strategies) and mutation-based diversification (~10 trajectories total), then applies per-trajectory reflection (Reflect(tᵢ, T)) and targeted rewrite (Revise(tᵢ, Rᵢ)) to correct logical inconsistencies and underdeveloped steps.

  2. Recombination: Cross-trajectory operators—crossover (splicing high-performing segments from trajectory pairs), transfer (injecting strategies from successful paths into weaker ones), and restructuring (global pool-level reorganization)—create genuinely novel hybrid trajectories.

  3. Refinement: A multi-dimensional reward function Reward(t,T) = α·TaskCompletion + β·ReasoningQuality + γ·Efficiency combining rule-based auto-evaluation and LLM-based expert evaluation selects elite trajectories for the next generation. Evolution terminates when reward improvement falls below threshold ε or iteration budget is exhausted.

The framework is plug-and-play on top of existing code agents (SWE-Agent/CodeAct used as the base).

Key Contributions

  • Trajectory-level self-evolution framework (revision + recombination + refinement) that generates genuinely diverse solution paths, not just surface-level sampling variants.
  • Demonstrates consistent gains across five LLMs spanning open- and closed-source families, validating model-agnostic applicability.
  • SoTA open-source result on SWE-bench Verified: 80.0% (Claude-4-Sonnet), 61.2% (Claude-3.7-Sonnet).
  • Ablation showing both revision and recombination are individually necessary; SE-Agent uniquely solves 12 issues unsolvable by all evaluated baselines.
  • Efficiency analysis showing near-optimal performance plateau at ~10 candidate trajectories.

Results

  • vs. SWE-Agent baseline (Pass@1): +112% relative (Llama-3.1-70B: 15.4%→32.6%), +80% (GPT-4o: 22.4%→40.4%), +51% (Claude-3.7-Sonnet: 40.6%→61.2%), +74% (Qwen-2.5-72B: 18.8%→38.8%), +73% (DeepSeek-V3: 31.6%→54.8%).
  • vs. SWE-Search (MCTS) baseline: ~+30% relative improvement on average across all five LLMs.
  • Pass@5 scores (SE-Agent): DeepSeek-V3 58.4%, Qwen-2.5-72B 42.4%, Llama-3.1-70B 35.2%, GPT-4o 44.8%, Claude-3.7-Sonnet 63.6%.
  • Leaderboard: 80.0% resolution rate with Claude-4-Sonnet on SWE-bench Verified (May 22, 2025 snapshot), vs. 66.6% for base SWE-Agent + Claude-4-Sonnet.
  • Ablation: Removing revision or recombination each individually degrades performance across all LLMs; removing both drops to SWE-Agent baseline level.

Limitations

  • Evaluated exclusively on SWE-bench Verified (software engineering / bug-fix domain); generalization to mathematical reasoning, planning, or embodied tasks is claimed but not demonstrated.
  • API cost scales with number of candidate trajectories; while near-optimal at ~10 trajectories, this still multiplies inference cost by ~10Ă— relative to single-trajectory agents.
  • Reward function hyperparameters (α, β, Îł) require task-specific tuning; sensitivity analysis is limited.
  • The 80.0% result bundles the latest SWE-Agent scaffold updates alongside SE-Agent's contribution, making it difficult to isolate SE-Agent's marginal gain at that operating point.
  • No wall-clock or total-token cost comparison against MCTS-based SWE-Search provided, making efficiency claims hard to verify.

Relevance to Agentic AI / LLM Agents

SE-Agent addresses a core challenge in agentic systems: the tendency of multi-step LLM agents to converge on locally optimal but globally suboptimal solution paths. The evolutionary trajectory manipulation paradigm is a direct alternative to MCTS for test-time search, requiring no gradient updates or reward model training, and can be layered on any ReAct-style agent as a plug-in module. The work contributes to the growing literature on test-time compute scaling and self-improvement loops, showing that cross-trajectory recombination—analogous to genetic crossover—yields qualitatively different search dynamics from temperature or prompt diversity alone. For practitioners building code agents or complex multi-step pipelines, SE-Agent establishes trajectory-level evolution as a practical and model-agnostic technique for pushing SoTA on hard benchmarks.