Skip to content

Monitoring Decomposition Attacks with Lightweight Sequential Monitors

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Neel Nanda/Andriushchenko; lightweight monitors catch decomposed harmful subtask attacks

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Decomposition attacks—where a harmful agent task is split into individually benign subtasks—achieve an 87% average attack success rate against GPT-4o, bypassing shallow safety alignment. The authors introduce a sequential monitoring framework that cumulatively evaluates prompt history, and show that prompt-engineered lightweight models (GPT-4o-mini, Llama-3.1-8B) can detect these attacks with up to 93% defense success rate while cutting cost by 90% and latency by 50% versus strong reference models.

Problem

Current LLM safety alignment is stateless and shallow: models only flag harm in the immediate prompt and cannot reason about long-range malicious intent spread across a multi-turn interaction. Prior work on decomposition attacks focused on the stateless QA setting, lacked released datasets of decomposed prompts, and proposed no robust defenses. This leaves agentic LLMs highly vulnerable to adversaries who decompose a single malicious goal into a sequence of seemingly benign subtasks.

Method

Dataset (DecomposedHarm). 4,634 harmful–benign task pairs are constructed across three domains: (1) general agent tasks (adapted from AgentHarm, 533 tasks across 8 harm categories), (2) text-to-image generation (631 tasks), and (3) question-answering (≈1,000 tasks). Harmful prompts are automatically decomposed into benign subtask sequences via few-shot prompting with a jailbroken LLM; a harmful index marks the step at which harm is realized if the LLM complies.

Sequential Monitoring Framework. After each new subtask prompt \(p_i\), an external monitor inspects the cumulative context \(C_i = [p_1, \ldots, p_i]\) and outputs a binary flag \(y_i \in \{0,1\}\). Flagging halts the underlying LLM immediately. True Positive is defined as flagging at or before the harmful index; False Negative is flagging only after it or not at all. The monitor is formalized as a binary sequence-labeling task evaluated via F1 and Defense Success Rate (Recall).

Prompt Optimization. A prompt-as-hyperparameter sweep on a unified validation set evaluates: zero-shot binary classification, in-context learning (ICL) with task-specific examples, chain-of-thought (CoT) reasoning, safety guidelines, and hypothesis generation. Monitors are prompted to return a probability of harmful intent; a threshold is tuned on the validation set. The best configurations are evaluated on held-out test sets using greedy decoding.

Key Contributions

  • DecomposedHarm dataset: 4,634 harmful–benign decomposed task pairs across agent tasks, text-to-image, and QA, with manually validated harmful indices; the largest and most diverse dataset for this threat.
  • Attack characterization: Demonstrates decomposition attacks are universally effective—GPT-4o's refusal rate drops from 50%→10% (agent tasks), 83%→2% (text-to-image), and ≈100%→9% (QA) after decomposition.
  • Sequential monitoring framework: An external cumulative-context monitor that intercepts harmful intent before the harm-realizing step.
  • Prompt-optimized lightweight monitors: Show that small models (GPT-4o-mini with QA-ICL, Llama-3.1-8B with agent-ICL) surpass expensive models and purpose-built guardrails, with substantially lower cost and latency.
  • Adversarial robustness: Monitors remain effective against random subtask injection, extreme obfuscation with massive subtask counts, and automated iterative red-teaming.

Results

  • Attack success rate (baseline): 87% average ASR on GPT-4o across all task types with decomposition attacks.
  • Optimized lightweight monitors vs. references (Table 1):
  • GPT-4o-mini (QA ICL): F1 = 0.918, cost \(1.39 \times 10^{-4}\)/task, latency 0.437s
  • Llama-3.1-8B (Agent ICL): F1 = 0.881, cost \(2.80 \times 10^{-4}\)/task
  • GPT-4.1-nano (CoT): F1 = 0.883
  • Reference o3-mini (zero-shot): F1 = 0.836, cost \(1.38 \times 10^{-3}\)/task, latency 3.976s
  • Reference GPT-4o (zero-shot): F1 = 0.870
  • Reference GPT-5 (zero-shot): F1 = 0.887, cost \(1.01 \times 10^{-2}\)/task
  • Guardrail baseline (Table 2): Llama-Guard-4-12B as a sequential monitor achieves macro F1 = 0.262 vs. optimized GPT-4o-mini's 0.926 (+0.664 improvement); Llama-Guard scores near 0 on QA and image tasks.
  • Cost/latency: Optimized GPT-4o-mini costs ~1/10th of o3-mini and ~1/3rd of GPT-4o; latency lower than all reference models.
  • Random subtask injection: Monitoring F1 degrades as injection rate increases, but lightweight monitors remain above reference model baselines at lower injection rates (Figure 4b).
  • Defense success rate: Up to 93% DSR reported for optimized lightweight monitors.

Limitations

  • The harmful index labeling relies on automated few-shot generation with manual spot-check validation (50 examples per dataset); mislabeled indices could inflate or deflate reported metrics.
  • Prompt optimization is task-specific (ICL examples from agent tasks generalize better for Llama, QA examples for GPT-4o-mini), so the optimal configuration may not transfer uniformly across deployment domains.
  • The monitoring framework does not address adversaries who operate across separate sessions or use side channels—it requires all subtasks to pass through the same monitored context window.
  • The paper is truncated before fully reporting adversarial red-team results (section 5 is cut off), so the extent of robustness under the most aggressive automated red-teaming conditions is not fully captured here.
  • QA tasks remain the hardest: even the best reference model (Claude-3.7-Sonnet) achieves only F1 = 0.669 zero-shot on QA, indicating residual vulnerability.
  • Open-source monitor cost estimates use Together AI pricing, which may not reflect all deployment scenarios.

Relevance to Agentic AI / LLM Agents

This paper directly addresses a critical security gap in agentic LLM deployments: the assumption that per-turn safety alignment suffices is demonstrably false when an adversary can distribute harm across a multi-step tool-use trajectory. The sequential monitoring framework is a practical, externally composable defense layer that does not require modifying the underlying agent model, making it architecturally compatible with existing multi-agent pipelines. The finding that lightweight models with prompt engineering outperform both heavyweight reasoning models and purpose-built guardrails on this task has important implications for designing cost-effective safety infrastructure in agentic systems. DecomposedHarm also serves as a benchmark for future work on stateful threat detection in long-horizon agent tasks.