Skip to content

Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs

🕒 Published (v1): 2025-12-04 11:00 UTC · Source: Arxiv · Venue: ACL 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MAMA (Multi-Agent Memory Attack) is a controlled evaluation framework measuring how network topology governs PII leakage in multi-agent LLM systems. Using a two-phase protocol (Engram seeding + Resonance extraction) across six canonical graph topologies, the study finds that denser connectivity, shorter attacker–target distances, and higher node centrality consistently increase PII leakage, regardless of model choice.

Problem

Multi-agent LLM systems lack systematic understanding of how communication topology shapes the PII leakage surface. Prior work on topology-aware security focused on adversarial content propagation and task degradation, not fine-grained privacy leakage; no controlled study isolated the effect of graph structure, attacker–target placement, and interaction rounds on memory extraction.

Method

MAMA models a multi-agent system as a directed graph G = (V, E) with three agent roles: a target agent seeded with private PII (Engram phase), an attacker using a subtle "information-gathering" framing (not overt theft), and benign collaborators. The dataset SPIRIT uses synthetic PII documents (104 entities, 25 background–question pairs) with strict sanitization: no PII appears in the shared public context, ensuring leakage must propagate through agent interaction. In the Resonance phase, agents execute up to 10 synchronous rounds of communication; at each round, each agent updates state based on its neighborhood N(v). Leakage is measured by a two-stage criterion: exact string match on attacker output, followed by LLM-based inference (DeepSeek-V3.1) on residual entities. Six topologies are tested (chain, circle, star-pure, star-ring, tree, complete) across n ∈ {4,5,6} agents and all non-redundant attacker–target placement pairs.

Key Contributions

  • MAMA framework: a reproducible, topology-controlled pipeline for measuring PII leakage in multi-agent LLMs with synthetic, sanitized data
  • Formal threat model with explicit agent roles (target, attacker, collaborators) and a unified leakage metric (LeakRate = fraction of PII entities recovered)
  • Empirical topology-leakage law: complete and star-ring are most vulnerable; chain and tree are most protective; the ranking is consistent across models and agent counts
  • PII-type taxonomy by "semantic resistance": spatiotemporal/location attributes leak most readily; regulated IDs (SSN, biometrics) remain near zero
  • Actionable design guidance: prefer sparse/hierarchical topologies, maximize attacker–target path length, restrict hub-bypassing shortcuts

Results

  • Topology effect (n=4, Llama-3.1-70B): complete = 29.65%, tree = 17.47%; (n=6) complete = 25.32%, chain = 12.95%
  • Topology effect (n=4, DeepSeek-V3.1): complete = 16.51%, chain = 11.91%; (n=6) complete = 18.70%, chain = 11.45%
  • Position effect (6-node circle, Llama-3.1-70B): adjacent pair (distance 1) = 29.49%; distance-2 pair = 15.38%; antipodal pair = 6.09%
  • Position effect (6-node chain, Llama-3.1-70B): distance-1 pair = 21.80%; distance-5 pair = 1.28%
  • Temporal dynamics: leakage rises sharply in rounds 1–3 and plateaus by rounds 3–4; most recoverable PII is extracted early
  • Model comparison: GPT-4o and GPT-4o-mini show substantially lower absolute leakage (≤5% vs. ≤30% for Llama-3.1-70B) but preserve topology-dependent ordering
  • PII type ordering: Spatiotemporal > Location ≥ Contact/Network ≥ Org-IDs ≫ Names > Regulated-IDs

Limitations

  • All experiments use synthetic PII; real-world leakage dynamics with actual sensitive data are not validated
  • Resonance horizon fixed at 10 rounds; longer interaction windows not explored
  • Single-attacker threat model only; coordinated multi-attacker scenarios excluded
  • Text-only communication; no tool use, code execution, or multimodal channels
  • Only six topology families; tree placements are subsampled to ≈1/3 of pairs for cost
  • LLM-judge inference step for leakage detection depends on DeepSeek-V3.1 reliability

Relevance to Agentic AI / LLM Agents

This paper directly addresses a foundational security concern for anyone building or deploying multi-agent LLM pipelines: private information seeded into one agent's memory can propagate to adversarial agents as a function of the communication graph, not just agent behavior. The finding that topology outweighs model choice as a leakage determinant has immediate implications for how agentic frameworks (e.g., AutoGen, LangGraph, CrewAI) should structure inter-agent communication, particularly in enterprise or healthcare deployments. The "Engram + Resonance" protocol is a reusable red-teaming primitive that security practitioners can apply to any multi-agent architecture. More broadly, the work bridges network science and agentic AI safety, framing agent graph topology as a first-order security parameter alongside prompt design and model alignment.