Stateful Online Monitoring Catches Distributed Agent Attacks¶

🕒 Published (v1): 2026-05-29 17:57 UTC · Source: Arxiv · link

Why this paper was selected

Stateful monitoring catches distributed agent attacks split across many sessions

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Standard LLM safety monitors score one agent transcript at a time, making them structurally blind to distributed attacks that split a harmful objective across many user accounts. This paper builds the first formal distributed agent attack, then defends against it with an online stateful monitor that uses stream clustering to aggregate weak suspiciousness signals across millions of transcripts and escalates only rarely to a cross-context LLM judge. The stateful monitor Pareto dominates standard monitors: 30% earlier detection at negligible added latency for ~99% of traffic.

Problem¶

Safety monitors assign a suspiciousness score to a single agent context, so an attacker who distributes a harmful task (e.g., network scan → credential harvest → lateral movement) across separate user accounts can keep each individual transcript below the detection threshold. Real-world examples (Anthropic 2025 threat disclosure) confirm this attack pattern is already deployed. The academic literature lacked both a formal attack model and a cross-account defense.

Method¶

Distributed attack (ARTEMIS-based scaffold): An orchestrator receives the full cybersecurity objective, then delegates narrow, reframed subtasks to subagents operating under fresh, innocent-looking contexts. Outputs are stored in a shared database; a verifier tool checks task completion. The monitor only sees subagent contexts.

Online stateful monitor (Algorithm 1): 1. Each incoming request \(x_t\) is scored by a standard monitor, yielding suspiciousness \(s_t \in [0,1]\). 2. \(x_t\) is embedded as \(e_t\); the nearest existing cluster centroid \(c_{j^*}\) is found via \(j^* = \arg\max_j \langle e_t, c_j \rangle\). 3. If \(\langle e_t, c_{j^*} \rangle < \tau_\text{route}\), a new cluster is opened; otherwise the centroid is updated online and \((x_t, s_t)\) is appended to the cluster buffer (deduplicated by hash). 4. If either the local score is elevated or the cluster buffer accumulates cross-session evidence (sum of top-\(k\) scores exceeds a calibrated threshold and \(\geq m\) prior contexts are above a minimum score), the cluster's top-\(k\) suspicious contexts are assembled and passed to a cross-context LLM judge that reasons across accounts.

Per-request cost is \(O(dM_t + M_t \log M_t) + q_t C_\text{cc}\), where \(M_t \ll N_t\) (number of clusters, not transcripts) and \(q_t \in \{0,1\}\) is rarely 1, keeping latency low for benign traffic.

Key Contributions¶

First fully general distributed agent attack: multi-agent scaffold (ARTEMIS) that completes hard Cybench tasks while distributing subtasks across accounts, evading standard monitors 5× more effectively than sequential decomposition.
Online stateful monitor algorithm using stream clustering (O(dM_t) per request) + a sparse cross-context LLM judge, enabling real-time cross-account detection.
Empirical evaluation against simulated datacenter traffic (272 attack sessions embedded in 2,720–27,200 benign conversations), showing Pareto dominance over standard monitors.
Two-month white-box red-team exercise (6 redteamers, >100,000 attacks, 2,985 sessions) that hardened the defense and revealed that stateful monitoring also catches standard jailbreaks because attackers reuse variants across accounts.

Results¶

Attack evasion: Distributed attack is caught only 20% as often as prior agent attacks at low FPR; enables >2× more harmful actions before detection vs. sequential decomposition.
Attack capability: gpt-oss-120b (0% solve rate on Cybench with standard scaffold) reaches 36.7–40.0% solve rate when orchestrating gpt-5.2 subagents with limited context.
Stateful monitor at 10× background: 81.1–87.2% campaign recall at 2.35–3.09% FPR vs. 75.0% recall at 2.72% FPR for standard monitor.
Stateful monitor at 100× background: 65.8% recall at 1.47% FPR vs. 55% recall for standard monitor at the same FPR.
Time-to-detect (TTD99) at 100× background: stateful monitor reaches 85.0% recall with TTD99 = 25.4 harmful steps at 4.30% FPR; standard monitor requires 9.01% FPR for the same recall and has TTD99 = 37.2.
Latency: ~99% of requests pay only local scoring + centroid lookup; tail latency (P99) is elevated, not the median.

Limitations¶

Detection advantage narrows as benign background traffic grows very large; the attack-to-benign ratio is a key parameter.
Defense assumes attack fragments leave repeated, semantically related evidence in embedding space; vocabulary substitution (renaming file paths, using different task terminology) was identified as a failure mode during red-teaming.
Monitor-context poisoning (prompt injection into the cross-context judge via crafted classifier output strings) was an additional red-team failure mode.
Evaluation uses simulated traffic built from existing coding-agent and web-chat traces, which may not fully capture the diversity of real production traffic.
Results depend on the specific embedding model (Voyage AI) and LLM judge (gpt-4.1-mini); generalization across providers is not evaluated.

Relevance to Harnesses / Meta-Harnesses¶

This paper directly instantiates a meta-harness for multi-agent safety: the stateful monitor is itself an orchestration layer that wraps individual agent sessions, aggregates their outputs, and applies a higher-level reasoning step across sessions—exactly the architectural pattern of a meta-harness. The attack scaffold (ARTEMIS with an orchestrator delegating to subagents with scoped contexts) is a concrete example of a task-decomposition meta-harness applied adversarially, demonstrating that harness-level decomposition can be weaponized to defeat per-agent safety checks. For researchers building or evaluating agent harnesses, this work establishes that safety and observability must be designed at the harness level (cross-context, stateful), not only at the individual agent level, and provides a stream-clustering algorithm for doing so efficiently.