Skip to content

Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction

🕒 Published (v1): 2026-07-02 06:27 UTC · Source: Arxiv · link

Why this paper was selected

Repo-scale vulnerability reproduction agent with strategy-grounded learning — strong SE+agent benchmark

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Mastermind is a dual-loop planner–executor framework for repository-scale vulnerability reproduction that separates transferable strategy learning from task-specific experience. A trainable Planner (Qwen3.6-35B-A3B) learns compact natural-language strategies via SFT and milestone-based GRPO, while frozen Executors (GPT-5.5, GPT-5.4 mini, GLM 5.1) instantiate those strategies as PoC rollouts. On CyberGym, this achieves 84.5% pass rate with GPT-5.5, versus 63.0% for Best-of-8 sampling.

Problem

LLM agents capable of executing repository-level commands still fail at strategic decisions—where to inspect, what input grammar to target, when to revise hypotheses. With a fixed GPT-5.5 executor, one-shot attempts solve only 23.5% of CyberGym tasks, exposing a planning bottleneck that better execution alone cannot fix. Existing approaches either learn high-level plans without preserving task-local memory, or store experience without updating the planner.

Method

Mastermind decomposes the problem into four components operating in two loops:

  1. Curator activates task-local experience (prior strategy/outcome records, failed hypotheses, verifier milestones) before each attempt and appends new records after verification.
  2. Planner (Qwen3.6-35B-A3B, trainable) emits a compact natural-language Strategy (capped at 2,000 tokens, median 700–1,100 tokens) describing where to inspect, what input structure to target, and how to validate the PoC. It does not issue shell commands.
  3. Executor (frozen) translates the Strategy into repository navigation, file edits, shell commands, and PoC submissions via CyberGym.
  4. Verifier scores each Rollout against CyberGym's milestone hierarchy (m0–m7).

Experience loop: verifier results update curator experience for the next attempt.
Policy loop: GRPO updates Planner weights using rewards from completed rollouts.

Training reward: $\(R(s, y) = \log\!\left(1 + r_{\text{milestone}}(m(y))\right) + \gamma_\ell f_\ell(s) + p_{\text{status}}(y)\)$ where \(f_\ell(s) = \mathbf{1}[s\text{ valid}]\max(0,\, 1 - n_s/T_{\text{strat}})\) penalizes trajectory-length drift, and \(p_{\text{status}} \leq 0\) penalizes timeouts. Advantages are normalized within same-task groups: \(\hat{A}_i = (R_i - \bar{R}_{\text{group}}) / (\sigma_{\text{group}} + \epsilon)\).

Slot-conditioned diversity: each 8-way GRPO group is seeded with eight distinct investigation biases (minimal-reproducer, parser-format, bounds-allocation, lifetime-state, etc.) to prevent strategy collapse.

SFT warm-starts the Planner on stepwise strategy transitions extracted from Claude Code and Codex trajectories, teaching it to revise rather than repeat.

Key Contributions

  • Strategy bottleneck identification: controlled ablations (Null Strategy 23.5% vs. Soft Oracle 39.5% M7 pass rate under identical executor/workspace) demonstrate strategy quality has a causal effect on reproduction success, independent of execution capability.
  • Dual-loop framework: separates Planner weight updates (transferable cross-task reasoning) from Curator experience (volatile task-local facts), assigning each knowledge type to the appropriate substrate.
  • Executor-agnostic transferability: a single trained Planner improves GPT-5.4 mini from 45.0% → 60.0% and GLM 5.1 from 58.5% → 71.0% without modifying either executor backbone.
  • Milestone-based GRPO: replaces a sparse binary success signal with an 8-level milestone schedule (m0–m7), enabling dense gradient for long-horizon RL over repository-scale tasks.

Results

  • GPT-5.5 executor: Mastermind 84.5% (169/200 tasks) vs. Best-of-8 63.0% (126/200), iterative improvement 77.0% (154/200), open-book PoC context 60.0% (120/200), one-shot 23.5% (47/200).
  • GPT-5.4 mini executor: Mastermind 60.0% vs. baseline 45.0%.
  • GLM 5.1 executor: Mastermind 71.0% vs. baseline 58.5%.
  • Strategy sensitivity (fixed GPT-5.4 mini): Soft Oracle 39.5%, Hard Oracle 32.0%, Zero-shot Planner 24.0%, Null Strategy 23.5%—showing that an executable strategy outperforms even ground-truth solution context provided directly.
  • Rollout efficiency: sequential iterative improvement reaches 154/200 with 753 rollouts vs. Best-of-8 at 126/200 with 1,600 rollouts; Mastermind further improves over this at comparable or fewer rollouts.
  • Training: 260-task split; evaluation: 200-task held-out split, stratified, seed 42.

Limitations

  • Evaluation confined to CyberGym (memory-safety/sanitizer crashes from ARVO and OSS-Fuzz); generalization to other SE task types is undemonstrated.
  • Executor rollouts carry a 900-second wall-clock timeout; no action- or token-count caps, making compute costs dependent on hosted service quotas and potentially difficult to reproduce exactly.
  • Hard Oracle (ground-truth solution provided) reaches only 32.0%—lower than Soft Oracle—suggesting the executor cannot always convert complete information into a working PoC, a residual execution bottleneck the framework does not address.
  • GRPO training is run over 260 tasks; scalability and data efficiency at larger task pools are not reported.
  • Strategy diversity relies on slot-conditioned prompting with eight fixed bias slots; this is a heuristic and may not adequately cover all vulnerability classes.

Relevance to Agentic AI / LLM Agents

Mastermind directly addresses the planning–acting split that is central to multi-step LLM agent design: it operationalizes the hypothesis that compact, intermediate-level plans are the correct learning unit for long-horizon agentic tasks, rather than full action trajectories. The demonstration that a single trained Planner transfers across frozen Executors speaks directly to modular agent architecture, where planning and execution can be upgraded independently. The dual-loop design—policy loop for cross-task generalization, experience loop for within-task memory—offers a concrete instantiation of how episodic and parametric memory should be separated in agentic systems. The slot-conditioned diversity mechanism and milestone-shaped reward are immediately applicable to other long-horizon agentic RL settings beyond cybersecurity.