Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents¶

🕒 Published (v1): 2026-07-09 17:26 UTC · Source: Arxiv · link

Why this paper was selected

Microsoft Research (Bo Peng); proactive memory surfacing for long-horizon agent trajectories

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Long-horizon LLM agents suffer from "behavioral state decay"—decision-relevant information (requirements, failed attempts, diagnoses) stops influencing behavior even when still nominally in context. This paper introduces a plug-and-play memory agent that runs alongside an unmodified action agent, maintaining a structured memory bank and selectively injecting reminders only when retained execution state is likely to change the next decision. Evaluated on Terminal-Bench 2.0 and \(\tau^2\)-Bench, it yields +8.3 pp and +6.8 pp pass@1 improvements respectively for Claude Sonnet 4.5.

Problem¶

LLM agents operating over long trajectories repeatedly violate requirements they identified earlier, retry failed commands, and re-diagnose error patterns they already resolved. The root cause is not that information is absent from the transcript, but that it stops exerting reliable control over behavior—a failure mode the authors term behavioral state decay. Existing memory systems address storage and retrieval but not when remembered state should re-enter the action loop; naive approaches (always inject or expose full bank) add latency and distract the agent.

Method¶

A two-agent architecture: an unmodified action agent \(\pi_A\) interacts with the environment; a separate memory agent \(\pi_M\) is invoked at step 1 and then at a fixed interval (every step in main experiments) with a sliding window of \(k=8\) recent messages plus the current memory bank \(B_t\).

\(\pi_M\) runs two sequential phases: 1. Phase 1 — Memory management: \(\pi_M\) emits structured tool calls (memory_update_status, memory_save_knowledge, memory_save_procedural, memory_delete) to maintain a three-component bank \(B_t = (s_t, K_t, P_t)\): a private status field \(s_t\) (never shown to the action agent), a knowledge store \(K_t\) (stable facts: requirements, env properties, file paths), and a procedural store \(P_t\) (attempts, failures, fixes, diagnostics). 2. Phase 2 — Intervention selection: Conditioned on the updated bank and recent trajectory, \(\pi_M\) either emits a concise targeted reminder \(r_t\) (injected as transient context into the next action-agent call) or the null action \(\emptyset\). The silence option is explicit and encouraged when no memory item is likely to affect the next decision.

Formally: \(B_t \sim \pi_M^{\text{edit}}(\cdot \mid x, w_t, B_{t-1})\); \(i_t \sim \pi_M^{\text{intervene}}(\cdot \mid x, w_t, B_t)\).

For open-weight memory policies, Qwen3.5-27B is first SFT-distilled from prompted memory-agent trajectories (SETA dataset), then fine-tuned with GRPO to calibrate intervention timing.

Key Contributions¶

Formalization of behavioral state decay as the central failure mode of long-horizon LLM agents, distinct from context-length or retrieval coverage issues.
A two-phase memory intervention architecture that decouples memory maintenance (what to retain) from intervention selection (when to reactivate), leaving the action agent fully unmodified (plug-and-play).
Empirical demonstration that selective proactive injection outperforms passive bank exposure, always-on injection, advisor-only guidance, and Mem0 retrieval baselines.
Preliminary evidence that the memory intervention policy is learnable by an open-weight model (Qwen3.5-27B via SFT+GRPO), with partial transfer to held-out Terminal-Bench.

Results¶

Terminal-Bench 2.0 (85 tasks, autonomous command-line execution):
Sonnet 4.5: 37.6% → 45.9% (+8.3 pp)
Opus 4.6: 43.5% → 45.9% (+2.4 pp)
\(\tau^2\)-Bench (278 tasks, interactive tool-use, task-weighted avg):
Sonnet 4.5: 55.0% → 61.8% (+6.8 pp); airline +10.0 pp, retail +9.6 pp, telecom +2.6 pp
Opus 4.6: 66.2% → 68.7% (+2.5 pp)
Ablations on \(\tau^2\)-Bench (Sonnet 4.5):
Full memory agent (ours): macro 64.3%, micro 61.2%
Full-bank context (no Phase 2 selection): macro 61.5%, micro 60.8%
Always inject (no silence option): macro 60.8%
Injection-only (no bank): macro 58.6%
Mem0 top-10 retrieval: macro 61.5%
Baseline: macro 57.5%, micro 55.0%
Selective intervention consistently outperforms all ablations; gains hold for stronger action agents, indicating the benefit is not merely compensating for weaker model capacity.

Limitations¶

The memory agent (Claude Opus 4.6) adds non-trivial inference cost at every step; cost analysis is not provided.
Fixed-interval triggering (every step) is used to isolate policy effects but is not necessarily optimal; smarter triggers (error-conditioned, context-shift-conditioned) are left as future work.
The open-weight training (Qwen3.5-27B SFT+GRPO) achieves only partial transfer to Terminal-Bench; full parity with the prompted Opus memory agent is not demonstrated.
Telecom domain shows only +2.6 pp gain for Sonnet 4.5, suggesting domain-specific dynamics that are not fully explained.
Evaluation is limited to two benchmarks; generalization to other long-horizon domains (e.g., ML engineering, web navigation) is not empirically validated.
The paper text is truncated before the full ablation discussion and the open-weight training section; some details may be incomplete in this summary.

Relevance to Agentic AI / LLM Agents¶

This paper directly addresses a fundamental bottleneck in deploying LLM agents on real-world long-horizon tasks: the reliability of state maintenance over extended trajectories. By framing memory as a selective intervention policy rather than a retrieval problem, it opens a new design axis—distinct from context compression, self-reflection, and retrieval-augmented generation—that is orthogonal to action-agent capability improvements. The plug-and-play nature means it can be layered onto existing agent harnesses without retraining the action model, making it immediately applicable to systems like SWE-agent, coding agents, and task-agent pipelines. The SFT+GRPO training path toward an open-weight memory policy is also relevant to the growing interest in modular, trainable agent components.