Memory Injection Attacks on LLM Agents via Query-Only Interaction

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MINJA is a memory injection attack against LLM agents that poisons an agent's long-term memory bank using only normal user queries—no privileged access required. By crafting queries with "bridging steps" and an indication prompt, an attacker induces the agent to autonomously generate and store malicious reasoning records. These records later mislead victim users' queries, achieving 98.2% injection success rate and 76.8% average attack success rate across three agent types.

Problem

Prior memory poisoning attacks (e.g., AgentPoison) assume the attacker has direct write access to the agent's memory bank or can inject triggers into other users' queries—an unrealistically strong assumption. No prior work addressed how to corrupt agent memory when the attacker is constrained to ordinary query-response interaction, the same access any regular user has.

Method

MINJA operates in two stages under query-only constraints:

1. Malicious record design via bridging steps: For a victim term v and target term t, MINJA constructs "bridging steps" bv,t (e.g., "Data of patient A is stored under patient B") that logically connect any benign attack query containing v to the target reasoning steps R_at for t. These bridging steps are prepended to the malicious output so that when the record is retrieved as a demonstration, the agent reproduces the same logic gap-filling.

2. Injection via indication prompts + Progressive Shortening Strategy (PSS): An indication prompt appended to the attack query induces the agent to autonomously generate bv,t and R_at as its output, which then gets stored in the memory bank. PSS iteratively removes segments of the indication prompt across successive attack queries, producing a sequence of malicious records with increasingly benign-looking queries. The final stored records resemble legitimate user queries, ensuring they are retrieved via cosine similarity when the victim later submits their query.

At inference, the victim's query retrieves these malicious records as in-context demonstrations, causing the agent to generate R_qt (target reasoning) instead of R_qv (correct reasoning).

Key Contributions

• MINJA: first memory injection attack requiring only query-level access, no direct memory manipulation or trigger injection into victim queries.
• Bridging step construction method to bridge the logic gap between benign victim-term queries and malicious target reasoning.
• Progressive Shortening Strategy (PSS) to iteratively strip indication prompts and populate the memory bank with retrievable malicious records.
• Empirical evaluation across three agents (EHRAgent, RAP, custom QA Agent) on four datasets (MIMIC-III, eICU, Webshop, MMLU) with four victim-target pair types, demonstrating 98.2% ISR and 76.8% ASR on average.
• Analysis of defenses showing all evaluated strategies (adversarial training, embedding sanitization, prompt-level detection, system-level isolation) have fundamental limitations against MINJA.

Results

• Injection Success Rate (ISR): 98.2% average across all configurations; exceeds 90% in nearly all per-pair settings.
• Attack Success Rate (ASR): 76.8% average; peaks at 98.9% for GPT-4o-based RAP on Webshop; lowest ~57% for GPT-4-based EHRAgent on MIMIC-III.
• Utility Drop (UD): Minimal (<2%) on MIMIC-III, eICU, and Webshop; -10% on MMLU (attributed to insufficient benign demonstrations retrieved).
• Retrieval noise robustness (Gaussian σ=0.01): ISR 100%, ASR drops only from 97.8% to 95.6%.
• Benign record density: ISR remains >90% up to 100 benign queries; ASR degrades on MIMIC-III (68.9%→31.1%) but stays >80% on eICU and Webshop.
• Prior-poisoned memory: ISR/ASR degrade 6–20 points but remain functional.
• Prompt-level detection: Targeted prompts catch 131/135 on MIMIC but 0/135 on RAP; general prompts improve cross-agent coverage but produce 34/50 false positives on EHR-MIMIC.

Limitations

• Attack effectiveness degrades as benign record density increases, particularly when retrieved demonstrations are structurally similar across users (e.g., short MIMIC-III queries).
• Prior poisoning by a different attacker reduces ISR and ASR by 6–20 points.
• ASR variance across victim-target pairs is high, reflecting pair-specific difficulty; some pairs achieve only ~30–40% ASR.
• Attack requires the shared memory bank assumption; isolated per-user memory banks would defeat it (though authors note identity-disguise circumvention).
• Small models (Llama-2-7B) lack sufficient task utility to make security evaluation meaningful, limiting generalization claims to capable LLMs.
• No evaluation on tool-augmented agents with external API side effects (only reasoning/retrieval chains tested).

Relevance to Agentic AI / LLM Agents

MINJA directly exploits the retrieve-then-reason architecture that underlies most practical LLM agents—the same design used in RAG-augmented agents, ReAct pipelines, and memory-enhanced assistants like ChatGPT's memory feature. It demonstrates that shared long-term memory is a novel attack surface qualitatively different from model weights or prompts: any unprivileged user can become a persistent threat to all future users of the same agent. For researchers building or securing agent systems, this establishes a realistic, low-barrier threat model that prior backdoor-attack literature underestimated, and it challenges the assumption that memory isolation or prompt filtering are sufficient defenses. The work also surfaces a fundamental design tension in multi-user agent deployments between cross-user memory sharing (for performance) and security isolation.