Skip to content

Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement

🕒 Published (v1): 2025-11-08 08:49 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SAGE (Self-Abstraction from Grounded Experience) is a three-stage test-time framework that enables LLM-based software-engineering agents to learn from their own rollout trajectories by inducing a structured plan abstraction, then conditioning a second execution on that plan. It achieves 74% Pass@1 on SWE-Bench Verified with OpenHands CodeAct + GPT-5, and 74.6% with an LLM-judge ensemble over heterogeneous SAGE configurations.

Problem

LLM agents tackling long-horizon software-engineering tasks (bug-fixing, repo-level code generation) operate within static execution frameworks: each task is attempted from scratch without principled reuse of prior rollout experience. Feeding raw trajectories back to the LLM is impractical because trajectories are long, noisy, and often exceed context limits, making it hard for the model to identify what matters. Existing alternatives—stepwise reflection and episodic reflection—fail to improve or even hurt performance on SWE-Bench Verified.

Method

SAGE instantiates three distinct agent roles over an MDP \(\mathcal{M} = (S, A, T, R, \gamma)\):

  1. Exploration (\(A_\theta\)): executes the task under the initial policy \(\pi_\theta\), producing trajectory \(\tau_\theta^{T_E} = (s_1, a_1, r_1, \ldots, s_{T_E}, a_{T_E}, r_{T_E})\).
  2. Plan Abstraction (\(P_\varphi\)): a (potentially different) LLM ingests \(\tau_\theta^{T_E}\) and emits a concise plan \(\psi \sim P_\varphi(\cdot|\tau_\theta^{T_E})\) structured around three pillars—Analysis (what the agent intended), Feedback (what went wrong), and Induced Plan (refined strategy).
  3. Plan-Augmented Execution (\(A_\theta^+\)): re-executes the same task under the revised policy \(\pi_\theta^+(\cdot|s_t, \psi)\), conditioning each action on the induced plan as additional context.

The plan \(\psi\) is framed as an option in the options framework: the initiation set \(I_o\) is the terminal state of exploration, the intra-option policy \(\mu_o\) is \(\pi_\theta^+\). All three roles can be instantiated with the same or different LLM backbones and agent frameworks (mini-SWE-Agent or OpenHands CodeAct), requiring only lightweight trajectory reformatting for cross-framework use.

Key Contributions

  • SAGE framework: a general test-time adaptation loop (explore → abstract → re-execute) that requires no weight updates and is model/framework-agnostic.
  • Plan abstraction as an option: formal grounding of the induced plan in the options framework, connecting self-abstraction to hierarchical RL.
  • Cross-model plan induction: empirical demonstration that using a different LLM for plan abstraction than for execution generally outperforms same-model self-assessment, mitigating self-bias.
  • LLM-judge ensemble: a lightweight single-call evaluator (Gemini 2.5 Pro) over heterogeneous SAGE configurations reaching 74.6% on SWE-Bench Verified.
  • Plan attribution analysis: quantitative evidence that the density of plan-surface traces in SAGE patches correlates with fail→pass outcome flips (34 vs. 17 pass→fail cases out of 500).

Results

  • Mini-SWE-Agent baseline vs. SAGE (SWE-Bench Verified, 500 instances):
  • GPT-5-mini (medium): 58.6% → 61.8% (+3.2 pp)
  • GPT-5 (medium): 65.0% → 67.2% (+2.2 pp)
  • GPT-5 (high): 66.6% → 71.4% (+4.8 pp)
  • Gemini 2.5 Pro: 52.4% → 53.6% (+1.2 pp)
  • Claude Sonnet 4: 62.0% → 64.0% (+2.0 pp)
  • Claude Sonnet 4.5: 72.0% → 72.4% (+0.4 pp)
  • Cross-model plan abstraction: Claude Sonnet 4.5 (actor) + GPT-5 high (planner) → 73.2%; GPT-5 high (actor) + GPT-5 high (planner) → 71.4% (same-model self-bias confirmed).
  • OpenHands CodeAct: Claude Sonnet 4: 68.4% → 71.6%; GPT-5 (high): 71.8% → 74.0%.
  • Ablation vs. alternatives (mini-SWE-Agent, GPT-5-mini subset): stepwise reflection 56.0%, episodic reflection 58.2%, 2Ă— scale LLM judge 59.2%, 2Ă— scale oracle 64.8%, SAGE 61.8%—SAGE surpasses all reflection baselines and the LLM-judge ensemble with equal compute.
  • Ensemble judge (5 heterogeneous SAGE configs, Gemini 2.5 Pro judge): 74.6% Pass@1; Best-of-5 oracle ceiling 83.4%.

Limitations

  • Each SAGE run doubles LLM calls (two full rollouts + one abstraction call), roughly tripling cost relative to a single-pass baseline.
  • Only validated on bug-fixing (SWE-Bench Verified); generalization to other long-horizon agentic tasks is asserted but not empirically demonstrated.
  • Plan abstraction quality depends heavily on the planner LLM; weaker planners can degrade performance below the same-model baseline (e.g., Claude Sonnet 4.5 planner on GPT-5 high actor: 68.2% vs. 71.4%).
  • Cross-framework use requires manual trajectory reformatting heuristics (eliding long outputs, inserting deterministic thought templates), which may not transfer cleanly to new frameworks.
  • The benchmark itself required custom patches to gold test cases (deprecated dependencies, transient HTTP 503 failures), complicating exact reproducibility against the official harness.

Relevance to Harnesses / Meta-Harnesses

SAGE is directly a meta-harness pattern: it wraps any underlying agent framework (mini-SWE-Agent, OpenHands) with an outer loop—explore, abstract, re-execute—without modifying the inner agent. This is structurally identical to a meta-harness that orchestrates multiple agent invocations, injects synthesized context between them, and routes execution across heterogeneous LLM backends. The cross-model plan abstraction stage (where a separate LLM judges and restructures the trajectory before feeding it to the actor) maps precisely onto the "judge/synthesizer" role common in multi-agent harness designs. The LLM-judge ensemble further demonstrates how a meta-harness can select among parallel agent outputs with a single lightweight evaluator call, a pattern directly reusable in broader harness orchestration. For researchers building harnesses, SAGE's result that same-model self-assessment degrades performance is a concrete, quantified argument for heterogeneous role assignment in multi-agent pipelines.