Skip to content

Toward Trustworthy Agentic AI: A Multimodal Framework for Preventing Prompt Injection Attacks

🕒 Published (v1): 2025-12-29 15:54 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper proposes a Cross-Agent Multimodal Provenance-Aware Defense Framework that wraps LangChain/GraphChain pipelines with layered sanitization and output validation agents to prevent prompt injection attacks from propagating through multi-agent graphs. A shared provenance ledger tracks token- and patch-level trust scores across modalities and agent hops. The system achieves 94% injection detection accuracy while retaining 96% task accuracy on benign inputs.

Problem

Existing prompt injection defenses (keyword filtering, safety fine-tuning, post-hoc output filtering) protect only single model boundaries and fail against paraphrased, visually encoded, or cross-agent-propagated injections. In multi-agent LangChain/GraphChain workflows, a single compromised upstream agent can contaminate all downstream nodes; no prior system maintains end-to-end provenance across modalities and agent hops simultaneously.

Method

The framework inserts four cooperating agents around every LLM call in a LangChain/GraphChain graph:

  1. Text Sanitizer Agent (At): Tokenizes input, runs a RoBERTa-based PIClassifier per span to compute injection intent scores, assigns trust scores via TrustModel, rewrites/removes override-intent spans, and records a provenance map P[ti] = {source, trust}.
  2. Visual Sanitizer Agent (Av): Extracts overlay text via PaddleOCR, reads EXIF metadata, computes CLIP patch embeddings, runs a StegoDetector for anomalies, assigns per-patch trust scores Ï„j, and redacts low-trust patches.
  3. Provenance Ledger: A Redis-backed key-value store aggregating text and visual provenance (modality, trust score, span/patch index, inter-hop influence), materialized into a trust-aware attention mask applied before LLM inference.
  4. Output Validator Agent (B): Scans LLM output for policy violations and secret leakage, computes influence attribution AttributionModel(Y, L) against the ledger, and either approves the output or triggers regeneration with tighter masks.

Sanitization occurs twice per agent node: once at graph ingress (global) and once immediately before the LLM call (pre-LLM). The system requires extending LangChain's BaseMessage, PromptTemplate, input interceptors, agent executors, tool/MCP executors, memory modules, and GraphChain node routers.

Key Contributions

  • Dual-stage sanitization (pre-agent ingress + pre-LLM) applied at every node in the agent graph, not just at the system boundary.
  • Unified cross-modal provenance ledger propagating token- and patch-level trust metadata across all agent hops.
  • Trust-aware attention masking that attenuates low-trust spans/patches at LLM inference time without blocking legitimate inputs.
  • Output validation layer with influence attribution that prevents low-trust content from driving LLM outputs and unsafe tool/MCP calls.
  • Concrete LangChain/GraphChain integration points and a Python implementation using RoBERTa, PaddleOCR, CLIP, and Redis.

Results

  • Detection accuracy: 94% (vs. keyword filtering 52%, post-hoc output filtering 61%, safety fine-tuning 66%, single VLM baseline unstated exact figure).
  • Cross-modal trust leakage: Reduced from 0.24 to 0.07 (70% reduction).
  • Task accuracy retention on benign inputs: 96% (vs. single-VLM baseline 94%).

Baselines: keyword filtering, safety fine-tuning, post-hoc output filtering, single VLM. No external benchmark dataset name is given; evaluation appears to be on internally constructed test cases.

Limitations

  • No named public benchmark dataset; experimental setup and adversarial test set construction are not described in detail, making reproducibility difficult to assess.
  • The PIClassifier and TrustModel are not fully specified; it is unclear how they generalize to novel jailbreak strategies beyond training distribution.
  • Prototype uses in-memory Redis; scalability and latency at production throughput are not measured.
  • Trust-aware attention masking modifies inference behavior but the paper does not measure the impact on model calibration or reasoning quality beyond aggregate task accuracy.
  • No ablation study isolating the contribution of each component (At vs. Av vs. provenance ledger vs. output validator).
  • The approach requires non-trivial modifications to LangChain/GraphChain internals, posing an integration burden for existing deployments.

Relevance to Harnesses / Meta-Harnesses

This paper directly addresses the security surface created by multi-agent harnesses: the trust boundary problem that arises when a meta-harness routes messages between specialized agents. The provenance ledger design—tracking trust metadata across every agent hop and enforcing sanitization at each node boundary—is an architectural pattern applicable to any orchestration harness (not just LangChain/GraphChain), and represents a principled approach to the "untrusted upstream agent" threat model that emerges in complex pipelines. For harness designers, the key insight is that output validation and provenance propagation must be built into the orchestration layer itself rather than delegated to individual agents or the LLM backend. The framework's integration points (message interceptors, prompt wrappers, output routers) map directly onto the hook/middleware abstractions common in meta-harness architectures.