Skip to content

AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AutoRedTeamer is a fully automated, end-to-end red teaming framework for LLMs that combines a dual-agent architecture (strategy proposer + red teaming agent) with a lifelong attack integration loop. It autonomously discovers new jailbreak techniques from research literature, implements them as Python plugins, and selects attack combinations via a memory-guided mechanism. It achieves 20% higher ASR on HarmBench against Llama-3.1-70B while reducing computational costs by 46% versus prior methods.

Problem

Existing automated red teaming tools either require human-curated seed prompts, optimize single attack vectors in isolation, or cannot incorporate emerging attack strategies without manual intervention. No prior framework combined attack discovery, implementation, memory-guided selection, and end-to-end prompt generation from high-level risk categories alone.

Method

AutoRedTeamer operates in two interleaved phases via a dual-agent design:

Strategy Proposer Agent: Queries Semantic Scholar and GitHub APIs for recent jailbreaking papers; scores each by novelty, distinctiveness from the existing library \(L\), and reported performance; implements promising attacks as Python classes inheriting a base attack interface; validates each on a HarmBench subset (threshold: ASR \(\geq 0.30\)); adds passing attacks to \(L\) with initial performance metrics.

Red Teaming Agent: Given a risk category or specific prompt \(U\), the pipeline is: 1. Risk Analyzer — decomposes \(U\) into testable sub-behaviors and scenarios. 2. Seed Prompt Generator — creates diverse test cases \(P\) in JSON format, replacing failures adaptively. 3. Strategy Designer — selects attack combinations from \(L\) using Attack Memory (embedding-based retrieval of similar past cases, running statistics per attack: ASR, query count, cost). 4. Attack Judge + Relevance Checker — scores outputs; triggers re-generation when prompts drift off-scope.

Attack success is measured as \(\text{ASR} = \frac{1}{N}\sum_{i=1}^{N} \text{JUDGE}(\text{LLM}(p'_i))\) under a black-box threat model. The memory system has three tiers: long-term (historical test cases + strategies), attack metrics (per-attack running statistics), and short-term (current test-case trajectory).

Key Contributions

  • Dual-agent architecture separating lifelong attack discovery (Strategy Proposer) from evaluation orchestration (Red Teaming Agent), enabling end-to-end operation from risk category strings alone.
  • Memory-guided attack selection combining embedding retrieval of similar past cases with per-attack and per-combination success-rate statistics — first in the field per the authors' comparison table.
  • Automatic attack implementation pipeline: paper → scored proposal → Python class → validated integration into \(L\), with 30% ASR validation gate.
  • Empirical demonstration of 20% higher ASR on HarmBench (Llama-3.1-70B) and 46% query reduction vs. strongest baselines; only method achieving nontrivial ASR on Claude-3.5-Sonnet (0.28 vs. near-zero for all others).
  • Coverage of all 314 AIR level-4 risk categories without human-curated prompts, matching diversity of the human-curated AIR-Bench benchmark.

Results

  • Llama-3.1-70B (HarmBench): AutoRedTeamer ASR = 0.82 vs. PAIR 0.60, TAP 0.60, AutoDAN-Turbo 0.67, Rainbow Teaming 0.18; uses 14 evaluation queries vs. 762 (TAP) or 8 + 60k dev queries (AutoDAN-Turbo).
  • GPT-4o: ASR = 0.69 vs. TAP 0.66, AutoDAN-Turbo 0.76; 16 queries vs. 683 (TAP).
  • Mixtral-8x7B: ASR = 0.96, matching AutoDAN-Turbo (0.96) and TAP (0.88) with far fewer queries (9 vs. 632).
  • Claude-3.5-Sonnet: ASR = 0.28; all baselines near zero (PAIR 0.04, TAP 0.05, AutoDAN-Turbo 0.02).
  • Attack integration overhead: 82 total queries for the proposer agent setup vs. ~60,000 for AutoDAN-Turbo and ~6,000 for Rainbow Teaming.
  • Generated test cases match diversity of human-curated AIR-Bench across all 43 level-3 AIR categories.

Limitations

  • Evaluation restricted to text-based LLMs under black-box constraints; does not cover multimodal or white-box settings.
  • Initial attack library seeded with only four human-written attacks; bootstrap diversity depends on this choice.
  • Attack integration agent uses Claude-3.5-Sonnet for code generation, creating a dependency on a proprietary model.
  • Validation gate (ASR ≥ 30% on HarmBench subset) may miss attacks that are effective only on specific target models or risk categories not well-represented in the validation set.
  • HarmBench evaluation uses GPT-4o as judge, introducing potential judge-model bias.
  • Paper is truncated before Section 4.3 results on risk category inputs are fully presented.

Relevance to Harnesses / Meta-Harnesses

AutoRedTeamer is a concrete instance of a meta-harness for adversarial evaluation: it wraps a library of attack modules, orchestrates their selection and combination via a memory system, and auto-extends the library by reading external research and synthesizing new plugins — precisely the "harness that manages harnesses" pattern. The lifelong integration loop (paper → proposal → implementation → validation → library update) is a self-extending meta-harness that parallels how software meta-harnesses dynamically register new tools or probes at runtime. The memory-guided strategy selection layer is directly analogous to learned configuration or hyperparameter-selection components seen in neural architecture search and AutoML meta-harnesses, applied here to attack orchestration. For researchers tracking harness design, the 30%-ASR validation gate and the structured Python base-class interface for attack plugins offer a transferable pattern for safely integrating dynamically discovered capabilities into an automated evaluation pipeline.