Skip to content

Data Leakage Prevention in Agentic Applications via Preemptive Hardening

🕒 Published (v1): 2026-07-21 08:35 UTC · Source: Arxiv · link

Why this paper was selected

Preemptive hardening against prompt-injection data leakage; actionable security for agentic deployment

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents a pre-deployment pipeline that statically scans agentic application source code, applies targeted hardening patches (schema tightening, allowlist gating, prompt rewriting, role separation), and validates the result against adversarial and benign inputs. Evaluated on five real-world CrewAI/LangGraph codebases and AgentDojo, the pipeline achieves 100% leakage elimination against basic jailbreaks and 91% reduction under stress-induced manipulation—without runtime policy enforcement.

Problem

Existing defenses against data leakage in LLM-based agents are either runtime-only (sandboxing, IFC monitoring) or prompt-based (defensive tokens, fine-tuning), neither of which addresses the recurring source-code-level weaknesses—unsafe I/O, over-privileged tool bindings, missing allowlists, credential exposure in prompts/logs—that are the actual root cause of most agent incidents. No prior approach prevents leakage in multi-agent workflows without degrading usability or requiring architectural overhaul.

Method

A three-stage pre-deployment pipeline operating exclusively on application source code:

  1. Analyzer: Parses Python ASTs to build agent topology \(A = (G, T, \mathcal{M}, E)\). Constructs a tool-parameter surface \(\Pi(t)\) with sensitivity function \(\rho : \Pi(t) \to \{\text{high, medium, low}\}\). An LLM with CoT reasoning propagates sensitivity labels \(\lambda(v) \in \{0,1\}^d\) via \(\lambda(v') = \lambda(v') \vee \bigvee_{v \in \text{pred}(v')} \lambda(v)\) across data-flow nodes, enumerating leakage paths \(\ell = \langle s, v_1, \ldots, v_k, \sigma \rangle\) and ranking sinks by \(R(\sigma) = \text{severity} \times \text{exposure} \times \text{sensitivity}\).

  2. Modifier: Applies targeted, local transformations at identified source-to-sink boundary crossings: prompt template rewriting (injecting stable control headers, demarcating untrusted text as data not instructions), unsafe context stripping (minimizing content crossing agent boundaries), role separation (typed handoffs, executor rejects instruction-like content), capability boundary enforcement (least-privilege tool allowlists, schema validators that block arguments with \(\lambda(v)>0\)), and outbound request transformation to scrub sensitivity labels before exfiltration sinks \(\sigma^\dagger\).

  3. Validator: Iteratively runs the hardened application \(A'\) over auto-generated adversarial inputs (jailbreaks, instruction overrides, tool-targeted manipulation) and benign task variants. Produces a structured security/utility report fed back to the modifier for further iteration.

The pipeline uses framework-specific adapters for CrewAI and LangGraph without requiring model retraining or runtime infrastructure.

Key Contributions

  • End-to-end CI/CD-oriented scanning-hardening-validation pipeline for multi-agent systems
  • Dependency-aware context construction via selective AST parsing that reconstructs agent functional surface across heterogeneous frameworks without intrusive refactoring
  • Hardening synthesis: a systematic set of patch templates covering allowlist gating, schema validation, argument sanitization, and least-privilege exposure, compiled into machine-verifiable guardrail invariants
  • Automated post-hardening validation that checks both security (leakage blocked) and utility (benign behavior preserved), enabling iterative remediation loops

Results

  • 100% reduction in data leakage against basic jailbreak and instruction-override attacks across all five real-world agentic applications
  • 91% reduction in leakage under stress-induced manipulation (more adaptive adversarial conditions)
  • Evaluated on five CrewAI/LangGraph codebases and the AgentDojo benchmark
  • Patches integrated without disrupting intended application behavior (confirmed by benign utility validation)
  • No continuous runtime policy enforcement required to achieve these results

Limitations

  • Evaluation covers only two frameworks (CrewAI, LangGraph); generalization to others (AutoGen, custom orchestrations) is claimed but not empirically demonstrated
  • The 91% (not 100%) residual leakage under stress conditions is not fully explained—specific failure modes are not detailed in the excerpt
  • LLM-augmented semantic reasoning in the analyzer introduces dependence on the backing LLM's labeling accuracy; hallucination in label propagation could produce false negatives
  • Static path enumeration may miss dynamically constructed leakage paths not visible at analysis time
  • No reported latency or compute cost of the pipeline itself, which matters for CI/CD integration at scale
  • Threat model assumes a black-box adversary with no source-code access; white-box adversaries are out of scope

Relevance to Agentic AI / LLM Agents

This work directly addresses the security engineering gap that grows as LLM agents move from prototypes to production: the mismatch between rapidly expanding tool-use capabilities and the lack of principled, automated pre-deployment hardening. For researchers tracking agentic AI, it provides a formal model of multi-agent leakage paths with compositional label propagation and a practical pipeline that complements runtime defenses (IFC, sandboxing) rather than replacing them. The emphasis on framework-agnostic, minimal-intervention patching is particularly relevant as the agent framework ecosystem (LangChain, AutoGen, CrewAI) remains fragmented, and the automated validation loop is a useful template for safe deployment of agents operating over sensitive user data.