Skip to content

PLAGUE: Plug-and-play Framework for Lifelong Adaptive Generation of Multi-turn Jailbreaks

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR; PLAGUE: plug-and-play lifelong adaptive multi-turn jailbreak generation framework

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

PLAGUE is a three-phase plug-and-play meta-framework for automated multi-turn LLM jailbreaking that combines a plan initialization module, an adversarial context-priming module, and a swappable finisher module, all augmented by a lifelong-learning vector-embedding memory of successful strategies. It achieves state-of-the-art attack success rates (81.4% SRE on o3, 67.3% on Claude Opus 4.1) by composing existing attacks (GOAT, Crescendo, ActorBreaker) as interchangeable sub-components rather than building a monolithic attacker.

Problem

Existing multi-turn jailbreaks lack four critical properties simultaneously: (1) lifelong learning — they cannot accumulate knowledge across attack runs and suffer from static strategy sets; (2) quality plan initialization — attacks like Crescendo and GOAT begin without structured, goal-anchored plans, causing semantic drift over turns; (3) modularity — components are tightly coupled, preventing systematic ablation or improvement of individual phases; (4) efficiency — the trade-off between attack success rate (ASR) and query budget is poorly managed. Prior SOTA (Crescendo) achieves only 37.4% ASR on o3; ActorBreaker plateaus near 60%.

Method

PLAGUE orchestrates three sequenced, modular phases:

Planner Phase: Given a harmful goal \(p_i\) sampled from HarmBench, the Planner LLM generates an \(n\)-step attack plan. It first queries a vector memory bank \(R^{+}\) using cosine similarity between the current goal embedding (Qwen3-Embedding-0.6B) and stored successful-strategy embeddings (threshold 0.6, max 2 retrieved examples). Retrieved strategies are injected as in-context learning (ICL) examples. If fewer than 2 are retrieved, random sampling from the library fills the gap. Strategies store: name, definition, and exact question sequence that produced the jailbreak.

Primer Phase: The first \(n-1\) plan steps (the final step is withheld to leave room for the Finisher) are executed sequentially against the Target LLM \(T\). At each turn, a Rubric Scorer \(R\) (Qwen3-235B-A22B) scores the response on four dimensions — Compliance (2 pts), Practicality (2 pts), Level of Detail (2 pts), Relevance (4 pts) — for a 10-point total. If the score falls below 7/10, a backtracking module removes the offending turn from \(H_T\) (but retains it in \(H_A\)) and reflects feedback into the next Attacker query. A summarizer (Llama 3.1-8B) compresses prior history to manage context overflow.

Finisher Phase: The Attacker receives the frozen primed context \(H_T\) and the original goal \(p_i\), then generates a final attack query. Refusals trigger backtracking with a relaxed rubric threshold of 3/10 (vs. 7/10 in Primer). The attack terminates on score \(> 8/10\) (success) or budget exhaustion. GOAT and Crescendo can substitute here as drop-in Finisher modules.

Lifelong Learning: On success, the Planner's strategy (name, definition, exact question sequence from \(H_t\)) is embedded by goal and stored in \(R^{+}\), enabling retrieval for semantically similar future goals via cosine similarity.

\[\text{ASR}(J) = \frac{1}{P}\sum_{i=1}^{P} J(p_i, MT_i)\]

The official metric is \(\text{ASR}@K\) (analogous to Pass@K, \(K=2\)), selecting the highest-scoring attempt across \(K\) runs.

Key Contributions

  • Three-phase attack decomposition (Planner → Primer → Finisher) that cleanly separates plan initialization, context-building, and goal execution
  • First multi-turn jailbreak with a lifelong-learning memory: cosine-similarity retrieval over goal-indexed successful strategy embeddings
  • Plug-and-play modularity: GOAT, Crescendo, and ActorBreaker's planner slot in as drop-in components with demonstrated performance impact
  • Fine-grained Rubric Scorer with backtracking enabling step-level reflection without corrupting Target history
  • Conversation summarization to prevent context overflow without catastrophic forgetting in the Attacker
  • Systematic ablation isolating contribution of each phase and the lifelong-learning component

Results

  • o3 (OpenAI): 81.4% SRE-ASR, +32.14% over previous SOTA (Crescendo: 37.4%)
  • Claude Opus 4.1: 67.3% SRE-ASR, +40.2% over previous SOTA (using Crescendo as Finisher; GOAT as Finisher yields lower numbers, demonstrating swappability matters)
  • Deepseek-R1, GPT-4o, Llama 3.3-70B: up to 97.8% SRE-ASR within 6 turns
  • Plugging ActorBreaker planner into PLAGUE: +15% diversity improvement with negligible ASR drop
  • ActorBreaker baseline: 30–90% lower ASR than PLAGUE across evaluations despite higher diversity
  • All results on HarmBench 200-sample standard set, averaged over 3 runs at \(K=2\)

Limitations

  • Attacker model (DeepSeek-R1) is large and expensive; compute budget implications are acknowledged but not fully quantified
  • Strategy library bootstrapped with only 2 human-authored Crescendo-derived strategies — warm-start dependency not fully characterized
  • \(K=2\) averaging partially masks variance from the high branching factor of multi-turn trajectories; true single-run reliability may be lower
  • Evaluated only on HarmBench's 200-sample standard set; generalization to other harm taxonomies or domain-specific targets is untested
  • Black-box API access is required for the Target; no analysis of rate-limiting, cost-per-goal, or detectability of the attack pattern
  • Paper text is truncated before full ablation tables, so some component-level comparisons are cited but not fully visible

Relevance to Harnesses / Meta-Harnesses

PLAGUE is architecturally a meta-harness: it wraps existing attack systems (GOAT, Crescendo, ActorBreaker) as interchangeable sub-components within a fixed orchestration shell, exactly analogous to how software meta-harnesses abstract over lower-level modules while adding cross-run state. The lifelong-learning memory — accumulating successful sub-strategies across runs and retrieving them for future orchestration decisions — is a direct instantiation of the persistent state + retrieval-augmented dispatch pattern central to production meta-harnesses. For researchers building harnesses, PLAGUE demonstrates how separating plan generation, context conditioning, and execution into distinct phases with well-defined interfaces enables modular improvement of individual stages without redesigning the whole pipeline; the backtracking + reflection loop within each phase also models a reusable adaptive retry with logged feedback primitive.