InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution

🕒 Published (v1): 2025-11-20 03:04 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

InfCode is an adversarial multi-agent framework for automated repository-level software issue resolution that iteratively co-refines test patches and code patches through a competitive loop between a Test Patch Generator and a Code Patch Generator, followed by a Selector agent that picks the best candidate. Using Claude 4.5 Sonnet as the backbone, it achieves 79.4% on SWE-bench Verified, a new state-of-the-art as of November 2025.

Problem

Existing agent-based and pipeline-based SWE systems generate tests that are too weak, misaligned with issue semantics, or insufficiently comprehensive, allowing code patches to "pass" verification while leaving the underlying defect unresolved. Test generation and code generation are treated as loosely coupled steps with no mechanism to force the verification signal to tighten over iterations.

Method

InfCode has two stages executed inside a per-issue Docker container:

Stage 1 – Adversarial Iterative Patch Generation. A Test Patch Generator and a Code Patch Generator engage in a bounded adversarial loop. The Test Generator creates/strengthens test cases to expose faulty behavior; the Code Generator rewrites the patch to pass them; when the patch passes, the Test Generator re-analyzes the issue to find gaps and adds harder tests. The loop terminates either when a hard iteration cap is reached or when the code passes all tests and the Test Generator cannot strengthen them further.

Stage 2 – Patch Selection. A Selector agent collects all candidate (code patch, test patch) pairs generated during Stage 1, evaluates them against metrics (functional correctness, test coverage, execution success, repository compatibility), runs them empirically in the container, filters overfitted solutions, and outputs the single most reliable patch.

The tool suite exposed to agents inside the container: Bash Tool (shell execution), Editor (fine-grained file edit), Searcher (ripgrep-based code search), Submitter (git diff extraction), and Executor (Docker I/O bridge).

Key Contributions

• Adversarial iterative co-refinement loop that simultaneously improves test quality and patch robustness, forcing both toward a dynamic equilibrium.
• Repository-aware containerized tool suite enabling reproducible, isolated patch generation and evaluation.
• Selector agent that performs empirical cross-validation across all generated candidate patches, providing a stronger final filter than single-pass methods.
• SOTA result of 79.4% on SWE-bench Verified (rank 1 as of 2025-11-14) and 40.33% on SWE-bench Lite with DeepSeek-V3.

Results

SWE-bench Lite (DeepSeek-V3 backbone): - InfCode: 40.33% (121/300) — best among all DeepSeek-V3 and GPT-4o baselines - KGCompass (DeepSeek-V3): 36.67% (110/300) - Agentless (GPT-4o): 32.00% (96/300) - SWE-agent (GPT-4o): 18.33% (55/300) - InfCode solved 11 more unique problems than KGCompass

Ablation on SWE-bench Lite (DeepSeek-V3): - Full InfCode: 40.33% (121) - w/o Adversarial Iteration: 36.33% (109) — −4.0 pp - w/o Selection: 32.33% (97) — −8.0 pp (Selection contributes more than adversarial loop alone)

SWE-bench Verified (Claude 4.5 Sonnet backbone): - InfCode: 79.4% (397/500) — rank 1 - TRAE + Doubao-Seed-Code: 78.80% (394) - Atlassian Rovo Dev: 76.80% (384) - EPAM AI/Run Developer Agent: 76.80% (384)

Tool invocation failure rates (per successful solve): - Bash Tool: 10.04% (highest; caused by malformed python -c calls and missing commands) - Editor: 5.80% - Searcher: 0.22% - Submitter: 0.00%

Limitations

• The Test Generator can produce tests that deviate from the issue description (adversarial "overthrow"), misleading the Code Generator into fixing the wrong behavior.
• Bash Tool and Editor still exhibit non-trivial failure rates; tool usage instructions need refinement.
• Evaluation is restricted to Python repositories; generalization to other languages is asserted but not demonstrated.
• SWE-bench Lite tests may be insufficiently rigorous, making it harder to filter incorrect patches (motivates the Verified subset experiments).
• No explicit mechanism to detect or recover from a Test Generator that consistently produces invalid or out-of-scope tests.

Relevance to Harnesses / Meta-Harnesses

InfCode is directly relevant as a harness architecture that orchestrates multiple specialized agents (Test Generator, Code Generator, Selector) within a containerized execution environment — precisely the kind of meta-harness pattern where a coordinating layer manages tool invocation, iteration control, and selection across sub-agents. The adversarial loop and hard iteration cap are harness-level control-flow decisions, not agent-level ones, illustrating how a meta-harness can enforce convergence guarantees. The containerized tool suite (Bash, Editor, Searcher, Submitter, Executor) represents a reusable harness substrate that is issue-agnostic, closely analogous to how meta-harnesses abstract environment setup and I/O routing from task-specific logic. For researchers tracking harness design, InfCode provides a concrete, benchmarked example of adversarial iteration as a harness-level strategy for driving agent quality upward without requiring stronger individual models.