Skip to content

MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents

🕒 Published (v1): 2026-07-16 07:19 UTC · Source: Arxiv · link

Why this paper was selected

Memory poisoning via normal interaction channels; persistent cross-session attack surface with concrete exploits

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MemPoison is a benchmark and mechanistic analysis framework for persistent memory poisoning in LLM agents, covering 1,227 hand-validated cases across a three-tier threat taxonomy. It shows that while write-time defenses suppress direct injections, they structurally fail against compositional (multi-record) and context-triggered dormant attacks. The authors introduce Mechanistic Influence Decomposition (MID) to explain why this defense frontier exists.

Problem

Prior work on agent memory security focuses narrowly on single-record injection and one memory substrate at a time, providing no unified framework to characterize how poisoning scales across compositional or dormant threat regimes. Existing write-time defenses (consistency checks, anomaly filters) lack evaluation beyond direct corruption settings, leaving the security community without a mechanistic explanation of when and why these defenses fail.

Method

MemPoison-Bench formalizes each test case as a tuple \(c = (C, I, Q, A_{\text{clean}}, A_{\text{poisoned}})\) — base context, adversarial injection, trigger query, and paired ground-truth behaviors — spanning four attack target types (fact, preference, instruction, state), three injection channels (user input, tool return, cross-agent), and three memory substrates (flat chunk, fact store, hierarchical notes).

The three-tier threat taxonomy encodes escalating write-time detectability: - L1 (Single Record): malicious intent is directly observable at write time. - L2 (Compositional): malicious payload is partitioned across multiple individually benign records; harm manifests only upon co-retrieval. - L3 (Context-Triggered Dormant): injected record is inert under normal retrieval; activation requires a specific natural trigger context.

Mechanistic Influence Decomposition (MID) provides counterfactual attribution over the retrieved memory set \(M = \{m_1, \ldots, m_k\}\): - Single-record influence: \(\Delta^s_i = d(r,\, r_{-i})\) - Pairwise interaction signal: \(\Omega^g_{ij} = \Delta^s_i + \Delta^s_j - d(r,\, r_{-ij})\) (positive value signals emergent compositional harm) - Activation shift: \(\text{ActivationShift}_i = \Delta^{\text{trigger}}_i - \Delta^{\text{normal}}_i\) (large positive value = dormant activation)

Key Contributions

  • MemPoison-Bench: 1,227 hand-validated cases across L1/L2/L3 difficulty levels, 4 attack types, 3 injection channels, and 3 memory substrates, constructed from 100 semantic units expanded to 12,760 raw cases and filtered via human review.
  • Cross-model evaluation: Behavioral Corruption Rate (BCR) and Clean Accuracy measured on 7 open-weight (Qwen3-8B/32B, Llama3.1-8B, Qwen2.5-14B, DeepSeek-V2-Lite/V3, GLM4-32B) and 3 closed-weight (GPT-4o, GPT-5, Gemini-3 Flash) models.
  • Defense frontier characterization: Systematic evaluation of 12 defenses (write-time, retrieval-time, anomaly, prompt-injection, sanitization) revealing a difficulty-dependent suppression ceiling.
  • MID framework: First structural explanation for write-time defense failure under L2/L3, decomposing memory influence into \(\Delta^s\), \(\Omega^g\), and ActivationShift components.

Results

  • Undefended BCR: Average 62.55% across all models; individual model range 54.67% (GPT-4o) to 66.87% (GPT-5); all models maintain CleanAcc >91%.
  • Difficulty gradient (undefended): L3 BCR (76.72%) > L2 BCR (51.73%) > L1 BCR (45.37%); L1–L2 gap is moderate because L2 requires joint retrieval of multiple fragments.
  • Write-time consistency check: Reduces L1 BCR from 45.37% → 4.77%, but L2 BCR only 22.54% and L3 BCR only 27.80% — confirming the structural blind spot.
  • Best overall defense (MIXed ensemble): BCR 10.70% overall, but L3 BCR remains 14.16%; CleanAcc preserved at 93.77%.
  • Memory substrates: flat_chunk most vulnerable (intact record preservation); fact_store most robust (decomposition dilutes malicious updates); hierarchical_notes intermediate.
  • Injection channels: cross_agent and tool_return produce higher BCR than user_input, identifying externally mediated writes as the critical attack surface.
  • PPL-based filter: Severely degrades CleanAcc to 61.28% (UR spikes to 31.57%), making it impractical.

Limitations

  • The benchmark excludes parametric poisoning, side-channel attacks, and base-model weight tampering — purely external memory lifecycle.
  • L2 BCR is retrieval-completeness limited: success requires all poisoned fragments to be co-retrieved, which may understate real-world risk if retrieval is more aggressive.
  • MID is a diagnostic tool, not a defense; it explains failure modes but does not propose a mitigating mechanism beyond motivating adaptive, context-sensitive defenses.
  • Evaluation is limited to text-based memory substrates; multimodal or graph-based memory architectures are not covered.
  • The threat model assumes no access to model weights or system prompts — insider or supply-chain threat scenarios are out of scope.

Relevance to Agentic AI / LLM Agents

Persistent memory is now a core architectural component in long-horizon LLM agents (task managers, personal assistants, multi-agent pipelines), and MemPoison directly characterizes the security cost of that design choice. The L2/L3 threat classes are particularly important for multi-agent systems, where cross-agent messages constitute the highest-risk injection channel. The MID framework provides a principled diagnostic vocabulary — \(\Delta^s\), \(\Omega^g\), ActivationShift — that the agent security community can use to audit memory retrieval pipelines and motivate the next generation of context-aware, adaptive defenses. The finding that no current write-time filter reliably suppresses dormant or compositional attacks is a direct call to action for agentic system designers who rely on RAG-style memory backends.