Skip to content

CoP: Agentic Red-teaming for Large Language Models using Composition of Principles

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CoP (Composition-of-Principles) is an agentic red-teaming framework that automates LLM jailbreak generation by having a Red-Teaming Agent orchestrate combinations of human-provided attack principles. It achieves state-of-the-art single-turn attack success rates while requiring up to 17.2× fewer queries than baselines. The framework is training-free, model-agnostic, and transparently auditable.

Problem

Existing automated jailbreak methods suffer from one or more critical deficiencies: GCG requires white-box gradient access and high compute; PAIR/TAP lack strategic guidance and generate unwieldy search spaces; AutoDAN-Turbo requires expensive per-target reinitialization of strategy libraries, making it cost-prohibitive on commercial APIs. All baselines achieve low ASR on strongly aligned models (e.g., ≤36.6% on Llama-2-7B-Chat). No existing framework allows principled, extensible, and interpretable composition of human red-teaming knowledge into automated attack pipelines.

Method

CoP consists of three LLM roles—Red-Teaming Agent, Target LLM, and Judge LLM—and a modular Principle Inventory of human-authored jailbreak strategies (e.g., Generate, Expand, Phrase Insertion, Style Change).

Pipeline (Algorithm 1): 1. Seed generation: Transform the raw harmful query \(q\) into an initial jailbreak prompt \(P_{\text{init}}\) using a fixed prompt template that bypasses the Red-Teaming Agent's own safety refusals. 2. Principle composition: The Red-Teaming Agent selects and combines one or more principles from inventory \(L\) using operator \(\oplus\) (e.g., Expand \(\oplus\) Phrase Insertion), producing a composed strategy. 3. Prompt refinement: The chosen CoP strategy is applied to the current best prompt \(P^\star\) to produce \(P_{\text{CoP}}\). 4. Dual evaluation: Judge LLM scores the Target LLM's response on jailbreak effectiveness \(s \in [1,10]\) and semantic similarity to original query \(\sigma \in [1,10]\). 5. Iterative refinement: If \(s_{\text{new}} \geq \eta\), return \(P_{\text{CoP}}\); if \(\sigma \leq \tau\), discard (prompt drifted); otherwise update \(P^\star\) if \(s_{\text{new}} > s^\star\). Repeat up to \(N\) iterations.

The framework requires only black-box access to the Target LLM and is plug-and-play: agent and judge LLMs can be swapped without retraining.

Key Contributions

  • Unified extensible harness: A Composition-of-Principles design that organizes attack knowledge as modular, composable principles; new exploits are added as new principles without pipeline retraining or reinitialization.
  • State-of-the-art ASR: Achieves 71–77% ASR on open-source Llama/Gemma models; 72.5% on Llama-2-70B-Chat vs. <50% for all baselines; up to 19.0× improvement over the best known single-turn ASR.
  • Query efficiency: Requires up to 17.2× fewer queries to the Target LLM than baseline methods.
  • Interpretable strategy attribution: CoP exposes which principle compositions drove successful jailbreaks (e.g., Expand alone: 12%; Expand \(\oplus\) Phrase Insertion: 9.8%; Expand \(\oplus\) Style Change: 6.0%).
  • Training-free, black-box generality: No gradient access or per-model strategy pretraining; applicable to both open-source and commercial LLMs.

Results

  • Llama-2-70B-Chat: CoP 72.5% ASR vs. all baselines <50%.
  • Llama-3-8B-Instruct-RR (safety-enhanced via Refusal Regularization): CoP 52% ASR despite reinforced guardrails.
  • Open-source range (Llama-2/3, Gemma): 71–77% ASR; 2.0–13.8× higher than GCG-T/PAIR/TAP/PAP-Top5; 1.1–2.2× higher than AutoDAN-Turbo.
  • Closed-source models tested: GPT-4-Turbo, Gemini Pro 1.5, O1, Claude-3.5-Sonnet (specific ASRs not included in the provided excerpt).
  • Query overhead: Up to 17.2× fewer target LLM queries than baselines.
  • Evaluation dataset: 400 HarmBench queries, judged by the HarmBench classifier (fine-tuned Llama-2-13B).
  • Baseline comparisons: GCG-T, PAIR, TAP, PAP-Top5, Rainbow Teaming, AutoDAN-Turbo.

Limitations

  • The provided text truncates before disclosing full closed-source model ASR numbers and ablation details.
  • CoP's Red-Teaming Agent (Grok-2 default) can itself trigger "Direct Refusal" on 16/400 HarmBench queries, requiring a seed-generation workaround that adds pipeline complexity.
  • The Principle Inventory (7 principles) is manually curated by humans; coverage depends on the quality and breadth of human-authored principles.
  • Requires access to a capable, non-safety-restricted Red-Teaming Agent LLM, which may have cost or access constraints.
  • Semantic similarity threshold \(\tau\) may cause premature resets if the target requires significant prompt transformation.
  • Evaluated only on single-turn jailbreaks; multi-turn threat models are out of scope.

Relevance to Harnesses / Meta-Harnesses

CoP is a direct instantiation of a red-teaming meta-harness: it does not implement a fixed attack but instead orchestrates a configurable inventory of attack sub-routines (principles) through an agentic control loop with feedback, scoring, and iterative refinement—the same architectural pattern as general-purpose meta-harnesses. The plug-and-play design (swappable agent/judge LLMs, additive principle inventory, no retraining) exemplifies modular harness composition, directly analogous to how task harnesses plug in evaluators or tools. For researchers building harnesses, CoP's dual-judge (effectiveness + semantic fidelity) and its adaptive best-prompt tracking are reusable design patterns for any iterative, agent-driven evaluation loop.