Skip to content

From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space

🕒 Published (v1): 2026-07-07 03:47 UTC · Source: Arxiv · link

Why this paper was selected

Reframes memory from passive retrieval to structured action space; novel architecture for conversational agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

NapMem reframes long-term user memory from a passive retrieval artifact into a structured action space that an agent learns to navigate via tool calls. A four-level memory pyramid (raw conversations → typed records → topic tracks → user profile) is exposed through five tools, and a GRPO-trained policy selects which level to query based on the evidence gathered so far. A 9B RL-trained model outperforms much larger untuned variants and all memory-system baselines on three user-memory benchmarks.

Problem

Existing user-memory systems treat memory access as a system-level retrieval function: the agent receives pre-selected context and has no mechanism to inspect additional evidence when retrieval is incomplete or mismatched to the query. This passive interface limits the agent's ability to handle heterogeneous memory requirements across personalization, long-range recall, and cross-session reasoning within a unified framework.

Method

NapMem builds a multi-granularity memory pyramid per user from interaction history, organized bottom-up into four levels: (1) raw conversations (append-only, highest fidelity), (2) memory records (typed atomic facts/events/instructions/preferences, reconciled incrementally via hybrid retrieval), (3) topic tracks (cross-session narratives over evolving user topics, stored as Markdown), and (4) user profile (stable global summary). Adjacent levels are linked by provenance identifiers, enabling both top-down and bottom-up traversal.

Five tools expose the pyramid: get_conversations, search_conversations, get_records, search_records, and read_files. At inference, the agent follows a sequential decision process:

\[a_t \sim \pi_\theta(\cdot \mid q, \mathcal{M}, a_{<t}, o_{<t})\]

producing trajectory \(\tau = (a_1, o_1, \ldots, a_k, o_k, y)\), capped at four tool calls.

The navigation policy is trained with GRPO using a rule-based terminal reward over three binary criteria—format validity \(F\), answer correctness \(C\), and memory-tool usage \(U\):

\[r(\tau) = \begin{cases} -1 & F=0 \\ 1 & F=1, C=1, U=1 \\ 0 & F=1, C=1, U=0 \\ -0.5 & F=1, C=0, U=1 \\ -1 & F=1, C=0, U=0 \end{cases}\]

Group-relative advantage \(A_i = r_i - \frac{1}{|G|}\sum_j r_j\) is applied uniformly to all output tokens, jointly optimizing answer quality and tool-selection behavior.

Key Contributions

  • Formulates long-term user-memory access as active memory navigation — a learnable, structured action space rather than a fixed retrieval pipeline.
  • Introduces the four-level provenance-linked memory pyramid covering raw evidence through stable user profiles with hybrid vector/keyword indexing on lower levels and Markdown files on upper levels.
  • Defines a five-tool interface for controlled, query-adaptive navigation across memory granularities.
  • Trains the navigation policy end-to-end with memory-tool RL (GRPO), requiring no per-level supervised labels.
  • Demonstrates that RL calibrates memory-tool use: reduces spurious memory calls on non-memory tasks (34.51% → 6.90% on GPQA-D) without degrading reasoning or function-calling performance.

Results

  • NapMem-9B w/ RL achieves the highest average score of 62.74 across LoCoMo, LongMemEval, and PersonaMem-v2 (vs. 59.85 for NapMem-397B, best baseline Mem0 at 59.25).
  • On LongMemEval: F1 57.41, L-J 80.33 — best among all systems.
  • On PersonaMem-v2: Acc. 47.97 — best among all systems.
  • On LoCoMo: F1 41.28 (best), L-J 59.92 (best).
  • Non-memory preservation: GPQA-D accuracy 57.58 vs. base 53.03; BFCL-v3 task accuracy 95.00 (matches base); V*Bench answer accuracy 91.10 vs. base 84.82.
  • Storage: NapMem uses 4.83 GiB total vs. 10.44 (Mem0), 14.10 (Zep), 23.10 (MemOS), with only AgeMem (2.99 GiB) smaller — but AgeMem scores far lower.
  • Inference: NapMem shows lower latency and fewer completion tokens than all baselines on PersonaMem-v2 (100-sample comparison); targeted navigation reduces generation length relative to passive-retrieval approaches that reason over large pre-fetched contexts.

Limitations

  • Tool-call budget is capped at four steps; queries requiring deeper traversal are counted as incorrect, which may penalize complex multi-hop memory tasks.
  • Each retrieval operation returns at most five items, which could miss relevant evidence for broad queries.
  • Memory pyramid construction requires an LLM-driven incremental pipeline; construction quality (record extraction, track merging, profile updates) is not separately benchmarked.
  • RL training uses only LoCoMo and PersonaMem-v2 queries; generalization to other memory domains (e.g., workplace, healthcare) is untested.
  • Ablation table is referenced in the text but the paper excerpt is truncated, leaving the quantitative contribution of upper pyramid levels and RL training partially unreported.
  • All baselines and NapMem use the same base LLM (Qwen3.5-9B), so the comparison isolates memory architecture choices but does not control for implementation quality across systems.

Relevance to Agentic AI / LLM Agents

NapMem directly advances the agentic memory subfield by demonstrating that memory access is itself a learnable skill — not just a retrieval engineering problem — and that RL over tool-use trajectories can teach a model when and at what granularity to consult memory without degrading general capabilities. This is a concrete instance of the broader shift toward agents that manage their own cognitive resources (context selection, evidence sufficiency judgment) rather than consuming whatever the system pipeline delivers. The provenance-linked pyramid design and the calibration result (near-zero spurious memory calls on non-memory tasks) are directly applicable to long-running personal assistant agents where indiscriminate memory retrieval would add latency and noise. The work also connects to the RL-for-tool-use thread (Search-R1, RMM, AgeMem), extending it from retrieval-query optimization to full navigation-policy learning over a structured multi-level store.