Skip to content

Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution

🕒 Published (v1): 2025-12-11 14:40 UTC · Source: Arxiv · Venue: ACL 26 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ReMe is a dynamic procedural memory framework for LLM agents that replaces passive memory accumulation with a closed-loop lifecycle: fine-grained experience distillation, context-adaptive retrieval/rewriting, and utility-based pruning. It achieves state-of-the-art on BFCL-V3 and AppWorld, and demonstrates a memory-scaling effect where smaller models with ReMe outperform larger memoryless models.

Problem

Existing agent memory systems treat the experience pool as a static, append-only archive. This causes three compounding failures: coarse-grained trajectory-level storage introduces irrelevant noise; retrieved experiences are applied verbatim without adaptation to shifted contexts; and no pruning mechanism removes degraded or outdated entries, polluting the pool over time.

Method

ReMe operates in three coordinated phases:

  1. Experience Acquisition: A summarizer LLM analyzes sampled trajectories (N=8 per task) via three analyses—success pattern recognition, failure analysis, and comparative analysis of success/failure pairs—to distill keypoint-level structured experience records E = ⟨ω, e, κ, c, τ⟩ (usage scenario, content, keywords, confidence, tools). Experiences are validated by LLM-as-a-Judge and deduplicated via cosine similarity before being indexed by usage-scenario embeddings in a vector database.

  2. Experience Reuse: At task time, top-K=5 experiences are retrieved by embedding similarity, optionally reranked by an LLM evaluating task-specific relevance, then rewritten into a cohesive task-specific guidance document before being injected as in-context demonstrations.

  3. Experience Refinement: Selective addition stores only experiences from successful trajectories; failed tasks trigger a failure-aware reflection loop (max 3 retries) where lessons from failure guide a new attempt, with the resulting successful trajectory added if the retry succeeds. Utility-based deletion removes any experience with retrieval count ≥ α=5 and utility ratio u/f < β=0.5.

Key Contributions

  • ReMe framework integrating multi-faceted distillation, context-adaptive reuse (retrieve → rerank → rewrite), and utility-based lifecycle management into a closed procedural memory loop.
  • Empirical demonstration of a memory-scaling effect: Qwen3-8B + ReMe surpasses vanilla Qwen3-14B; Qwen3-14B + ReMe surpasses vanilla Qwen3-32B.
  • Release of reme.library, a fine-grained procedural memory dataset with structured success patterns and failure lessons across diverse agentic tasks.

Results

  • Qwen3-8B + ReMe (dynamic) vs. No Memory baseline: +8.83% Avg@4, +7.29% Pass@4 averaged across BFCL-V3 and AppWorld.
  • Memory-scaling: Qwen3-8B + ReMe (dynamic) Pass@4 = 55.03% vs. Qwen3-14B No Memory = 54.65%; Qwen3-14B + ReMe (dynamic) Avg@4 = 44.66% vs. Qwen3-32B No Memory = 40.89%.
  • ReMe (dynamic) outperforms A-Mem and LangMem across all three model scales on both benchmarks; LangMem collapses on AppWorld for small models while ReMe remains consistent.
  • Ablation (Qwen3-8B, BFCL-V3): selective addition over full addition yields +3.50% Avg@4; adding failure-aware reflection and utility-based deletion yields further sequential gains to reach 45.17% Avg@4 / 68.00% Pass@4 from 40.83% / 62.00% baseline.
  • Keypoint-level vs. trajectory-level extraction: +4.17% Avg@4 gap on Qwen3-8B.
  • Computational overhead: +2.54 seconds average latency per task on AppWorld vs. No Memory baseline.
  • Generalized to 6 additional LLMs including GPT-4.1, o4-mini, Qwen3-Max, Kimi-K2-Thinking, DeepSeek-V3.2, GLM-4.7—consistent Avg@4 gains across all.

Limitations

  • Retrieval is performed once per task at the start; no mid-task dynamic retrieval, which may miss context changes during long multi-turn interactions.
  • Experience validation relies solely on LLM-as-a-Judge, which may miss nuanced quality signals.
  • Stronger summarizers yield disproportionate gains (Table 5), meaning performance depends partly on summarizer capacity; small-model summarization remains a bottleneck.
  • Failure-aware reflection is capped at 3 retries to avoid infinite loops, which may leave some recoverable failures unexplored.
  • Evaluated only on tool-augmented benchmarks (BFCL-V3, AppWorld); generalization to open-ended or embodied domains is untested.

Relevance to Agentic AI / LLM Agents

ReMe directly advances the procedural memory subfield of agentic AI by closing the loop between experience accumulation and active lifecycle management—a gap left open by prior work (AWM, Agent KB, CER) that focuses on extraction or retrieval but not pruning. The memory-scaling finding is particularly significant for the field: it shows that well-managed episodic/procedural memory can substitute for raw model scale, offering a compute-efficient path to lifelong agent improvement without retraining. The retrieve-rerank-rewrite pipeline and utility-based pruning are practical design patterns directly applicable to any tool-use agent architecture, making this a strong reference for practitioners building production agent memory systems.