PLAGUE: Plug-and-play Framework for Lifelong Adaptive Generation of Multi-turn Jailbreaks

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; PLAGUE: plug-and-play lifelong adaptive multi-turn jailbreak framework

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

PLAGUE is a plug-and-play, three-phase framework for automated multi-turn LLM jailbreaking that incorporates lifelong learning via a vector-embedding strategy memory. It achieves state-of-the-art attack success rates (ASR) of 81.4% on OpenAI o3 and 67.3% on Claude Opus 4.1 using only black-box API access, improving over prior SOTA by 32%+ with a comparable or smaller query budget.

Problem

Multi-turn jailbreaks are significantly more effective than single-turn attacks but lack systematic study of what makes them succeed. Existing methods (GOAT, Crescendo, ActorBreaker, RACE) each suffer from at least one of: semantic drift across turns, failure to learn from past attacks, tactically rigid strategy sets, or poor planning initialization — preventing them from reliably breaking strong safety-aligned models.

Method

PLAGUE decomposes every multi-turn attack into three sequentially chained phases:

1. Planner: Constructs an \(n\)-step attack plan using the goal \(p_i\) as a query and retrieves at most two past successful strategies from a vector memory \(R^{+}\) via cosine similarity (\(\tau = 0.6\)) between goal embeddings (Qwen3-Embedding-0.6B). These serve as in-context learning (ICL) examples for the Attacker LLM to generate a novel plan.

2. Primer: Executes the first \(n-1\) plan steps against the Target LLM, building adversarial context while appearing benign. A Rubric Scorer \(R\) evaluates each turn on four axes — Compliance (2 pts), Practicality (2 pts), Level of Detail (2 pts), Relevance (4 pts). Turns scoring below 7/10 trigger backtracking: the turn is removed from the Target's history \(H_T\) but retained in the Attacker's history \(H_A\) for reflection. A conversation summarizer provides short-term memory.

3. Finisher: With context frozen, the Attacker executes the final blow against the original goal. Scoring threshold relaxes to 3/10 for backtracking; a score \(>8/10\) marks success and triggers extraction of the winning strategy into \(R^{+}\) (indexed by goal embedding) for lifelong learning.

Existing methods (GOAT, Crescendo, ActorBreaker) slot in as drop-in replacements for the Planner or Finisher modules. The final reported metric is \(\text{ASR}@K\) (\(K=2\), best-of-two attempts), averaged over three runs.

Key Contributions

• A modular three-phase taxonomy (Planner → Primer → Finisher) that disentangles plan initialization, context-building, and goal delivery in multi-turn attacks.
• A lifelong-learning memory architecture using goal-embedding-indexed cosine retrieval (\(R^{+}\)) that accumulates successful strategies across attack objectives without catastrophic forgetting.
• A fine-grained Rubric Scorer (10-point, four-category) enabling adaptive backtracking and reflection at every turn.
• Plug-and-play composability demonstrated by swapping ActorBreaker's planner and GOAT/Crescendo as finishers without redesigning the framework.
• First multi-turn jailbreak to simultaneously feature lifelong learning, a reflection module, open-source strategy library, and back-tracking (Table 1).

Results

• 97.8% ASR (StrongReject) on Deepseek-R1, GPT-4o, and Llama 3.3-70B within 6 turns.
• 81.4% ASR on OpenAI o3; +32.14% over previous SOTA baseline.
• 67.3% ASR on Claude Opus 4.1 (with Crescendo as Finisher); +40.2% over previous SOTA.
• Crescendo alone achieves as low as 37.4% ASR on o3; ActorBreaker plateaus ~60% ASR across most models.
• PLAGUE outperforms all single-turn and multi-turn baselines across HarmBench 200-sample standard set.
• Swapping in ActorBreaker's planning module raises diversity by 15% with negligible ASR drop.
• Using GOAT as Finisher for Claude Opus 4.1 underperforms Crescendo-as-Finisher, illustrating the utility of module-level tuning.

Limitations

• Requires a query budget; performance is reported as ASR@2 (two attempts per goal) to manage variance from the high branching factor of multi-turn conversations.
• Strategy library is cold-started with only two seed strategies adapted from Crescendo; performance in very early attacks before the library grows is not separately characterized.
• Black-box only: no ablation of whether access to logits or gradients would close residual gaps.
• The Finisher phase's effectiveness is sensitive to the choice of Finisher module, requiring per-target model tuning.
• Paper text is truncated; full ablation on semantic drift metrics and longer-horizon attack budgets may contain additional caveats not visible here.

Relevance to Agentic AI / LLM Agents

Multi-turn dialogue is the default interaction paradigm in agentic LLM deployments, making PLAGUE's findings directly relevant to the security posture of any agent system: an adversarial user can spread harmful intent across seemingly benign conversational turns that individually pass safety filters. PLAGUE itself is architected as an agentic system — it combines reflection, episodic memory with embedding retrieval, conversation summarization, and backtracking, mirroring the building blocks of production LLM agents. The lifelong-learning component demonstrates that red-teaming agents can compound knowledge across goals, analogous to how production agents are expected to generalize across tasks. For researchers tracking agentic AI, this work surfaces a critical vulnerability class: safety evaluations conducted in single-turn settings dramatically underestimate risk in the multi-turn, long-context, tool-augmented sessions that agentic workflows produce.