Skip to content

PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents

🕒 Published (v1): 2026-07-21 08:49 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

PhoenixRepair is a multi-agent framework for automated GitHub issue resolution that addresses the tendency of LLM agents to over-commit to a single (often wrong) edit location and to under-explore repair attempts at each location. It samples multiple candidate edit locations, iteratively refines patches via reflection and selection, and distills historical guidance into a final-round generation step, achieving 76.0% Pass@1 on SWE-bench-Verified.

Problem

Existing agent-based repair methods suffer from two forms of insufficient exploration: (P1) probabilistic LLMs exhibit high-probability bias, causing repeated sampling to concentrate on the same (potentially incorrect) edit location; and (P2) a single repair attempt at the correct location is often insufficient—the generated patch may be incomplete, requiring iterative correction. Test-time scaling (multiple parallel samples) and trajectory-level mutation (SE-Agent) both fail when all samples converge on a wrong location.

Method

PhoenixRepair wraps any base agent scaffold (here, SWE-agent) in three sequential phases:

  1. Multi-location sampling: A localization agent runs \(N\) sequential iterations, each receiving the prior \(i-1\) locations as references to encourage diversity. Resulting locations \(L'\) are deduplicated (same file + entity + overlapping lines = duplicate). For hard tasks (\(|L'| > k\), default \(k=4\)), a graph-based localization agent (LocAgent) constructs a heterogeneous dependency graph (imports, invocations, inheritance) and extracts the top-\(l\) ranked entities via cross-iteration confidence estimation; these are merged: \(L_{final} = \text{dedup}(L' \cup L_{graph})\).

  2. Iterative reflection & refinement: A coder agent generates one patch per location in \(L_{final}\), producing \(w\) (trajectory, patch) pairs. A selection agent scores each pair on two axes: (a) \(Q_{test}(\tau_i)\)—reproduction and edge-test quality assessed on a compressed trajectory (repetitive/failed actions merged); and (b) \(R_{pass}(p_i)\)—regression test pass rate on a curated \(T_{final}\) (tests unrelated to the reported issue, filtered by a test agent). Top \(\lceil w/2 \rceil\) candidates survive; the loop repeats until one edit location remains.

  3. Final-round generation: An analysis agent distills all historical attempts at the surviving location \(\ell_{final}\) into guidance text, which is injected into the coder agent's system prompt for final patch generation. For simple tasks (\(|L'|=1\)), \(t\) parallel patches are sampled and the same distillation is applied.

Key Contributions

  • Multi-location sampling with sequential diversity injection and graph-augmentation for hard tasks, directly addressing location-level under-exploration.
  • Iterative selection loop with a two-dimensional quality metric (test quality + regression pass rate) that halves the candidate set each round.
  • Distilled-guidance injection for final-round generation, accumulating cross-attempt experience into a compact prompt signal.
  • Demonstrated plug-in generalizability: the mechanisms integrate into Mini-SWE-agent and Live-SWE-agent with consistent gains, not just SWE-agent.

Results

  • 76.0% Pass@1 on SWE-bench-Verified (500 tasks) under MiniMax-M2.5—highest reported among evaluated configurations.
  • 7.8% relative improvement over SWE-agent under DeepSeek-V3.1 (66.0% vs. 61.2%).
  • Relative gains over SWE-agent across all five backbones: +7.8% (DeepSeek-V3.1), +6.4% (Qwen-Coder-Plus), +7.2% (DeepSeek-V3.2), +6.4% (GLM-4.7), +5.3% (MiniMax-M2.5).
  • Relative improvements over Agentless range from +9.1% to +11.9% across all configurations.
  • Fault localization accuracy (DeepSeek-V3.2 vs. SWE-agent): File Acc@1 +3.13%, Module Acc@1 +3.50%, Function Acc@1 +5.08%.
  • PhoenixRepair outperforms SWE-Search (MCTS-based), OpenHands, Trae-agent, and Moatless-tools under all shared model configurations.

Limitations

  • Token and inference cost not fully quantified in the provided text (RQ5 is listed as a research question but results are truncated); the iterative loop and graph-based agent necessarily increase compute.
  • The graph-based localization agent (LocAgent) is an external dependency; performance on tasks where graph traversal fails or cross-file dependencies are sparse is not characterized.
  • Threshold hyperparameters (\(k\), \(N\), \(l\), \(t\), \(w\)) are fixed empirically (e.g., \(k=4\), \(N=5\), \(l=2\), \(t=3\)); sensitivity analysis is not shown in the provided text.
  • Evaluated only on SWE-bench-Verified; generalization to different programming languages or issue types beyond Python/GitHub is not assessed.
  • The deduplication criterion (same file + entity + overlapping lines) may aggressively merge locations that are semantically distinct at the line level.

Relevance to Harnesses / Meta-Harnesses

PhoenixRepair is structurally a meta-harness: it wraps existing agent scaffolds (SWE-agent, Mini-SWE-agent, Live-SWE-agent) with an outer orchestration layer that controls sampling strategy, candidate selection, and prompt injection without modifying the inner agent logic. The three-phase loop—sample, select, distill-and-rerun—is a canonical meta-harness pattern: an outer controller that repeatedly invokes sub-agents and aggregates their outputs to steer subsequent invocations. The distilled-guidance injection mechanism is particularly relevant to the meta-harness literature, as it demonstrates how a coordinator can compress cross-attempt signal into a durable context that improves subsequent agent calls, a form of in-context memory management at the harness level. Its plug-in design (RQ3) provides empirical evidence that meta-harness orchestration logic can be decoupled from the base scaffold, a key architectural property for modular harness composition.