Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

🕒 Published (v1): 2026-07-16 06:13 UTC · Source: Arxiv · link

Why this paper was selected

Franziska Roesner (UW security); memory-as-attack-surface; critical threat model for persistent agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Memory files in agentic systems (CLAUDE.md, AGENTS.md, behaviors.md) constitute a persistent prompt injection surface analogous to stored XSS: payloads planted in these files survive session boundaries and influence future behavior even when direct injection from untrusted external content rarely succeeds. Across Claude Code and Codex with four models, attack success rates vary substantially by model, adversarial goal, and attack vector, with no model uniformly resistant across all conditions.

Problem

Prior prompt injection research treats attacks as single-session phenomena; existing benchmarks like AgentDojo evaluate only within-session attack trajectories. As agentic systems adopt persistent file-based memory (CLAUDE.md, AGENTS.md, knowledge bases) that evolves across sessions, a new class of durable, cross-session compromise becomes possible — but has not been systematically studied. This work addresses the gap between within-session (reflected) and cross-session (stored) prompt injection in real agentic coding systems.

Method

The authors build a sandboxed synthetic workspace simulating a personal assistant for a fictional user "Jordan Kim," with a structured hierarchy of auto-loaded files (CLAUDE.md / AGENTS.md), referenced knowledge files (coding/knowledge/stack_preferences.md), and universal behavior files (core/behaviors.md). Three adversarial goals are evaluated — credential exfiltration (SSH key via Python telemetry snippet), unauthorized tool use (install PyYAML 5.3.1 with known CVEs), and brand targeting (recommend Amex card in financial queries) — each payload planted in the file where it would plausibly appear. Experiments follow a tuple protocol: (initial_state, benign_task, attack_payload, persistence_target, trial_number). Session sequences include: (1) single probe, (2) Probe → Stabilization → Probe with the same payload (persistence test), and (3) Probe → Stabilization → Probe with different payloads (stacking test). Attack Success Rate (ASR) and persistence rate are computed over 10 trials per condition across Claude Code (Haiku 4.5, Opus 4.7) and Codex (GPT-5.2, GPT-5.5).

Key Contributions

• Defines a cross-session persistent prompt injection threat model distinct from single-session injection, drawing an explicit stored-XSS analogy
• Constructs a controlled synthetic multi-domain workspace for replicable agent security evaluation
• Introduces multi-session experiment sequences (single probe, repeat, stacked) to measure payload persistence and compounding
• Provides empirical ASR and persistence-rate results across 2 agentic systems, 4 frontier models, 3 adversarial goals, and 3 attack vectors
• Distinguishes harness-managed memory (patched at platform level) from user-curated workspace files (transparent, version-controlled, intentionally agent-writable), showing the latter is a complementary and understudied attack surface
• Demonstrates that direct injection from untrusted external content rarely succeeds in overwriting trusted files, but pre-planted payloads in those files reliably affect behavior

Results

Single-probe ASR (10 trials per cell):

Model	Credential Exfiltration	Unauthorized Tool Use	Brand Targeting	Mean
Haiku 4.5	80%	100%	10%	63.3%
Opus 4.7	0%	90%	0%	30.0%
GPT-5.2	30%	0%*	40%	23.3%
GPT-5.5	0%	80%	100%	60.0%

*GPT-5.2's 0% on unauthorized tool use reflects an execution capability limitation, not a security check.

• Stronger Claude models (Opus) resist credential exfiltration and brand targeting better than weaker ones (Haiku), but unauthorized tool use still achieves 90% ASR on Opus
• GPT-5.5 achieves 100% ASR on brand targeting despite being the stronger Codex model, showing no monotonic relationship between capability and security across goals
• Both auto-loaded and referenced (conditionally loaded) files serve as viable attack vectors
• Multi-session persistence and stacking results are presented in the paper but were truncated in the provided text

Limitations

• Threat model assumes the payload is already in memory; how it arrives there (upstream injection, supply-chain, user import) is out of scope
• Synthetic workspace may not reflect complexity or diversity of real user environments
• Only 10 trials per condition; stochastic variance in agentic systems makes this a limited sample
• Restricted to file-based memory in local agentic systems; platform-managed web-chatbot memory is explicitly excluded
• Multi-session (persistence/stacking) quantitative results are not fully reproduced in the provided text excerpt
• No defenses are implemented or evaluated; the paper motivates future defensive work rather than proposing solutions

Relevance to Agentic AI / LLM Agents

This paper directly addresses a structural vulnerability that grows in severity as agents become more persistent and self-modifying: the memory mechanism that enables long-horizon usefulness is the same mechanism that enables durable compromise. The stored-XSS framing gives agent security researchers a concrete threat model for cross-session attacks, which is absent from existing benchmarks like AgentDojo. For practitioners building or deploying agentic systems with file-based memory (CLAUDE.md workflows, skill registries, knowledge bases), the finding that unauthorized tool use achieves 90% ASR on Opus 4.7 despite being a frontier model is a practical warning. The work motivates memory-specific defenses — integrity verification, trust-level tagging, or write-access controls on persistent files — as a necessary addition to the agentic security stack.