Skip to content

When Agents “Misremember” Collectively: Exploring the Mandela Effect in LLM-based Multi-Agent Systems

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; Mandela Effect in LLM MAS — collective misremembering phenomenon

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLM-based multi-agent systems exhibit a "Mandela effect"—agents collectively adopt socially-reinforced false memories that contradict verifiable facts. The paper introduces ManBench, a 4,838-question benchmark across five interaction protocols, demonstrating this vulnerability across 13 models and proposing mitigation strategies that reduce the effect by 74.40% on average.

Problem

Individual LLM hallucination and conformity are well-studied, but a distinct failure mode remains unaddressed: when multi-agent social interactions cause agents to collectively internalize false beliefs into stable long-term memory—analogous to the human Mandela effect. Prior work treats factual errors as stateless and ignores the memory-consolidation dynamics that make socially-induced misinformation durable. No benchmark existed to systematically measure this phenomenon.

Method

The authors construct ManBench from BIG-Bench Hard tasks, curating 4,838 multiple-choice questions across four domains (History/Events, Misconceptions & Social Cognition, General Knowledge, Domain-Specific Knowledge). For each question, a plausible distractor is selected as the implanted false target.

Five interaction protocols vary along two axes: - Group composition: Generic Group (undifferentiated agents build simple consensus) vs. Role-based Group (five agents with specialized roles: Error Conclusion Initiator, Detail Support Provider, Group Consensus Reinforcer, Authority Endorser, Questioning Compromiser). - Memory timescale: Short-term (subject agent responds within the same conversation context) vs. Long-term (a memory consolidation step distills the false narrative into a summary, which is then provided as context in a fresh conversation, simulating recall).

Three metrics quantify the effect: error rate \(\text{Err}^P\), reality shift rate \(\sigma^P = |Q^P_\times \cap Q^B_\checkmark| / |Q^B_\checkmark|\) (correct baseline memories overwritten), and maximal reality shift rate \(\sigma_{\max}\) (union of overwritten memories across all four influence protocols).

Mitigation strategies tested: prompt-level defenses (cognitive anchoring, source scrutiny) and a model-level alignment approach.

Key Contributions

  • ManBench: first benchmark specifically designed to evaluate the Mandela effect in multi-agent settings; 4,838 questions, 5 protocols, covering 4 knowledge domains.
  • Comprehensive empirical study across 13 LLMs (7 commercial, 6 open-source) showing universal susceptibility.
  • Quantitative analysis of five causal factors: group composition, memory timescale, group size, knowledge domain, and model scale.
  • Prompt-level and alignment-based defenses achieving average 74.40% reduction in the Mandela effect vs. baseline.

Results

  • All 13 LLMs are susceptible: Qwen3-235B's error rate rises from 25.48% (baseline) to 74.75% under Role-based Short-term protocol; GPT-5's error rate more than doubles from 17.63% to 41.59% under the same protocol.
  • Role-based group > Generic group: Claude 4 Sonnet's reality shift rate rises from 15.45% (\(\sigma^{GS}\)) to 35.21% (\(\sigma^{RS}\)); GPT-4o from 46.04% to 55.95%.
  • Long-term persistence is model-dependent: GPT-5 and Llama3.3-70B show strong self-correction (\(\sigma^{RL}\) drops to 1.67% and 19.75% respectively); Claude 3.5 Haiku and Qwen3 series retain false memories (Claude 3.5 Haiku: \(\sigma^{RS}=63.67\%\), \(\sigma^{RL}=55.63\%\); Llama3.1-8B: \(\sigma^{GL}=85.13\%\)).
  • Group size saturation (Generic): effect plateaus around 7 agents; GPT-4o-mini and Claude 3.5 Haiku reach ~56% \(\sigma^{RL}\) at that size.
  • Inverted-U for Role-based groups: effect peaks at 6 agents (\(\sigma^{RS}>63\%\)); for 9+ agents, overall error rate drops below the baseline Reality Protocol (below 32%), attributed to "suspicion-induced vigilance."
  • Mitigation: combined strategies reduce the Mandela effect by an average of 74.40% across models.

Limitations

  • ManBench is derived from BIG-Bench Hard (multiple-choice format), which may not capture open-ended or free-form memory distortion in real deployments.
  • Specious evidence is generated by prompting LLMs to produce plausible distractors—this is synthetic and may not reflect naturalistic adversarial social dynamics.
  • The memory consolidation step is a prompted summarization, not a persistent external memory store (e.g., vector database), limiting ecological validity for production systems with more complex memory architectures.
  • Analysis focuses on knowledge retrieval tasks; generalization to planning, code generation, or other agentic task types is not established.
  • Evaluation of group size stops at 15 agents; dynamics at larger scales (common in real deployments) are unexplored.

Relevance to Agentic AI / LLM Agents

This paper directly targets a systemic vulnerability in multi-agent LLM architectures: the social propagation and consolidation of false beliefs across agents, which degrades collective reasoning reliability in high-stakes deployments like fact-checking, contract review, and policy analysis. It operationalizes a previously informal concept (the Mandela effect) into a rigorous benchmark and causal framework, filling a gap distinct from both hallucination (individual, stateless) and sycophancy (short-term compliance). The finding that role-specialization—a core design pattern in agentic systems—actually amplifies misinformation susceptibility is a critical design-time warning for practitioners building debate, reflection, or consensus-based multi-agent pipelines. Proposed defenses (cognitive anchoring, source scrutiny, alignment) offer actionable mitigations applicable to existing agent frameworks.