Skip to content

G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✩ Ask Claude

TL;DR

G-Memory introduces a three-tier graph-based memory architecture (insight, query, and interaction graphs) that enables LLM-based multi-agent systems to accumulate and reuse cross-trial collaborative experience. It performs bi-directional traversal at query time to retrieve both high-level generalizable insights and fine-grained compressed interaction trajectories, then updates all three graph levels after each execution. Integrated as a plug-in across three MAS frameworks, it improves success rates by up to 20.89% on embodied action and 10.12% on knowledge QA without modifying the underlying frameworks.

Problem

Existing multi-agent systems (MAS) lack principled cross-trial memory mechanisms. Their memory is either absent, confined to inside-trial context, or limited to storing final outputs—ignoring the nuanced inter-agent collaboration trajectories. Directly porting single-agent memory to MAS fails because MAS trajectories are up to 10× longer and require role-specific, agent-customized memory cues rather than undifferentiated retrieval.

Method

G-Memory organizes MAS interaction history into three hierarchical graph structures: - Interaction Graph — directed graph of atomic utterances with temporal edges encoding which utterance prompted which response. - Query Graph — nodes are past queries (with status and pointer to their interaction graph); edges encode semantic relationships between queries. - Insight Graph — nodes are distilled generalizable insights abstracted from query outcomes; edges are hyper-connections linking co-relevant insights through shared supporting queries.

At retrieval, given a new query Q: (1) coarse-grained embedding similarity retrieves top-k semantically similar queries from the query graph, then 1-hop graph expansion enlarges this set; (2) upward traversal (query→insight graph) collects relevant high-level insights; (3) downward traversal (query→interaction graph) uses an LLM-based graph sparsifier to extract core subgraphs from the M most relevant historical interaction graphs; (4) a role-aware operator Ω distributes filtered insights and trajectory snippets to each agent according to its role.

At update, all three levels are written: a new interaction graph is traced, a new query node is added with edges to supporting queries and utilized insights, and new insight nodes are distilled from the completed trajectory and linked into the insight graph.

Key Contributions

  • Identification of the MAS self-evolution bottleneck as a memory architecture deficiency, distinguishing MAS requirements from single-agent settings.
  • G-Memory: a three-tier hierarchical graph memory (insight / query / interaction) with bi-directional retrieval and agentic, incremental update.
  • Role-specific memory augmentation: each agent receives memory filtered to its designated role.
  • Plug-and-play integration demonstrated across AutoGen, DyLAN, and MacNet without framework modification.
  • Token-efficient design: outperforms MetaGPT-M with ~36% fewer additional tokens on PDDL+AutoGen.

Results

  • ALFWorld (embodied action): G-Memory + MacNet + Qwen-2.5-14b achieves 79.10% vs. 58.21% no-memory baseline — +20.89%.
  • SciWorld: G-Memory + MacNet + Qwen-2.5-14b reaches 68.11% vs. 57.53% no-memory — +10.58%.
  • HotpotQA (knowledge QA): G-Memory + MacNet (GPT-4o-mini) achieves 35.69% vs. 28.57% no-memory — +7.12%; broader QA gains up to +10.12% noted in abstract.
  • Average across 5 benchmarks: G-Memory surpasses the best single/multi-agent memory baseline by +6.8% (AutoGen+Qwen-2.5-7b) and +5.5% (MacNet+Qwen-2.5-7b).
  • Token efficiency: On PDDL+AutoGen, G-Memory adds only 1.4×10⁶ tokens for 10.32% gain vs. MetaGPT-M adding 2.2×10⁶ tokens for 4.07% gain.
  • Ablation: removing fine-grained interaction module drops average by 4.47% (AutoGen) / 3.82% (DyLAN); removing insight module drops by 3.95% / 3.39%.
  • Sensitivity: 1-hop expansion and k∈{1,2} retrieved queries optimal; larger values degrade performance (e.g., k=5 causes −7.71% on ALFWorld+AutoGen).

Limitations

  • Validated on only three task domains (embodied action, knowledge QA, planning games); generalization to medical QA or other specialized domains untested.
  • Retrieval quality degrades with aggressive hop expansion or large k, suggesting fragility to hyperparameter tuning as memory grows.
  • Memory graph maintenance overhead (LLM-based sparsification and insight distillation) is not ablated for latency cost—only token cost is reported.
  • Evaluated with a limited set of MAS frameworks (AutoGen, DyLAN, MacNet); applicability to more architecturally diverse systems (e.g., code-execution agents, tool-use pipelines) is unverified.

Relevance to Agentic AI / LLM Agents

G-Memory directly advances the self-evolution capability of multi-agent systems—a recognized open frontier—by replacing flat or artifact-only memory with a structured, evolving knowledge graph that captures inter-agent collaboration dynamics. The role-specific memory augmentation addresses a core MAS challenge: heterogeneous agents need differentiated context, not uniform retrieval. The plug-in design means these gains transfer to existing frameworks (AutoGen, DyLAN, MacNet) without retraining or architecture changes, making it practically applicable to the current ecosystem. For researchers tracking MAS and long-horizon agent tasks, this paper establishes that memory architecture, not just reasoning or communication topology, is a first-class design concern for collective agent intelligence.