Skip to content

Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems

🕒 Published (v1): 2026-04-24 03:08 UTC · Source: Arxiv · link

Why this paper was selected

Context-fragmented violations: individually safe agent actions collectively break policies

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Multi-agent enterprise deployments create a new class of security failure called Context-Fragmented Violations (CFVs), where each individual agent action is locally policy-compliant yet the collective sequence violates organizational invariants. The paper formalizes CFVs mathematically, constructs a benchmark (PhantomEcosystem), and proposes Distributed Sentinel—a sidecar-based zero-trust architecture using provenance-tracked taint tokens—to enforce cross-domain policies without exposing raw inter-departmental data. The system achieves F1 = 0.95 at 106ms latency, compared to 0.85 for prompt-based filtering and 0.65 for rule-based DLP.

Problem

Existing alignment mechanisms (prompt-based guardrails, monolithic interceptors) operate on individual agents' local context. When organizational policies are siloed across departmental knowledge graphs, no single agent has the information needed to detect a violation—e.g., a marketing agent forwarding an R&D log containing an NDA-protected codename acts legitimately locally but violates global policy. Additionally, LLM summarization/paraphrasing causes semantic laundering: security constraints attached to surface text are stripped as content flows through transformation chains, even when semantic meaning is preserved.

Method

Distributed Sentinel wraps each agent \(A_i\) with a sidecar proxy \(S_i\) and introduces the Semantic Taint Token (STT) protocol:

  • Each outgoing message carries an encrypted token \(\text{STT} = \langle \texttt{src\_id}, \texttt{taint\_vec}, \texttt{constraints}, \texttt{sig} \rangle\), where taints attach to provenance (source graph node identifiers), not surface text, making them immune to paraphrasing.
  • When agent \(A_j\) prepares a sensitive action, \(S_j\) issues a Cross-Domain Predicate Query to the source sidecar \(S_i\): "Does node \(\texttt{src\_id}\) permit scope \(X\)?" The response is a single bit (True/False); no raw graph data crosses domain boundaries.
  • Before execution, Counterfactual Graph Simulation forks a copy-on-write snapshot of the local graph, applies the proposed mutation, and checks all local invariants against the hypothetical post-action state.
  • STT propagation satisfies \(\pi(\tau(d)) \supseteq \tau(\varphi(d))\) (Theorem 2.12): taint tracking is a conservative over-approximation—it may over-restrict but never misses a constraint through LLM transformations.
  • A Neuro-Symbolic Entity Mapper \(f: \text{Text} \times G_i \to 2^{V_i}\) grounds free-form agent outputs to graph nodes via NER + session provenance alignment + graph lookup.
  • Optional ZK-SNARK (Groth16) extensions allow zero-knowledge predicate proofs for environments requiring even stronger privacy than a 1-bit boolean response.

Key Contributions

  • Formal definition of Context-Fragmented Violations (CFVs) and proof of the Fundamental Limitation of Local Enforcement (Theorem 2.6): no local mechanism can reliably detect CFVs without cross-domain information.
  • Formal definition of Semantic Laundering (Definition 2.8: \(T(\varphi(t)) \subset T(t)\)) and proof that provenance-based STT propagation is laundering-resistant.
  • PhantomEcosystem benchmark: 200 scenarios (160 attack / 40 safe controls) across 9 CFV categories (direct leak, multi-hop, aggregation, time-series, side-channel, scope creep, data reconstruction, cross-org, token manipulation), average chain length 2.4, no explicit labels forcing graph-state reasoning.
  • Distributed Sentinel architecture with formal soundness, privacy-preservation (1 bit/query leakage), and tamper-resistance (Ed25519 EUF-CMA) proofs.
  • Empirical violation study of 8 frontier LLMs (GPT-5.4, GPT-4.1, Claude Opus 4.6, Gemini 2.5 Pro, and others): all show 14–98% violation rates; cross-domain flows structurally worse than same-domain.

Results

  • Distributed Sentinel: F1 = 0.95, end-to-end latency 106ms (16ms verification + 90ms entity extraction on A100).
  • Prompt-based filtering baseline: F1 = 0.85.
  • Rule-based DLP baseline: F1 = 0.65.
  • Ablation: removing STT propagation entirely degrades the system to F1 = 0.53, confirming STT is the primary load-bearing component.
  • All 8 evaluated LLMs fail at self-avoidance: violation rates range 14–98% across models; cross-domain flows show systematically higher rates than same-domain flows.

Limitations

  • Requires all security constraints to be codified in departmental knowledge graphs; constraints existing only in unstructured documents or tacit human knowledge are outside the model.
  • The completeness guarantee (Corollary 2.11) depends on the assumption that every security-relevant constraint resides in at least one \(G_i\)—a strong assumption that may not hold in practice.
  • Entity grounding via the neuro-symbolic mapper is a potential failure point: if NER or graph resolution errors occur, downstream STT propagation operates on incorrect symbolic state.
  • Benchmark (PhantomEcosystem) is synthetic; real enterprise KG structure and inter-agent communication patterns may differ substantially.
  • ZKP extensions are described but not empirically evaluated in the reported latency numbers.

Relevance to Harnesses / Meta-Harnesses

Distributed Sentinel is structurally a meta-harness: a cross-cutting enforcement layer that wraps arbitrary agent implementations via sidecar proxies without modifying agent internals, directly analogous to how meta-harnesses intercept and coordinate agent pipelines. The STT Protocol and Counterfactual Graph Simulation components implement the same logic that a harness needs when orchestrating multi-step agent workflows—tracking what state has flowed where and pre-validating proposed actions before committing them. The paper's core finding—that local, per-agent alignment is insufficient and a centralized enforcement layer operating above individual agents is necessary—provides theoretical grounding for the meta-harness pattern itself. The 9-category PhantomEcosystem taxonomy also offers a reusable checklist of cross-agent failure modes that harness designers should test against.