Skip to content

Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents

🕒 Published (v1): 2026-07-09 17:26 UTC · Source: Arxiv · link

Why this paper was selected

Proactive memory agent decouples memory from action; critical for long-horizon harness design

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Long-horizon LLM agents suffer from "behavioral state decay"—decision-relevant information (requirements, failed attempts, diagnoses) stops influencing behavior even when still nominally in context. This paper introduces a plug-and-play memory agent that runs alongside an unmodified action agent, maintaining a structured memory bank and selectively injecting reminders only when retained execution state is likely to change the next decision. Evaluated on Terminal-Bench 2.0 and \(\tau^2\)-Bench, it yields +8.3 pp and +6.8 pp pass@1 improvements respectively for Claude Sonnet 4.5.

Problem

LLM agents operating over long trajectories repeatedly violate requirements they identified earlier, retry failed commands, and re-diagnose error patterns they already resolved. The root cause is not that information is absent from the transcript, but that it stops exerting reliable control over behavior—a failure mode the authors term behavioral state decay. Existing memory systems address storage and retrieval but not when remembered state should re-enter the action loop; naive approaches (always inject or expose full bank) add latency and distract the agent.

Method

A two-agent architecture: an unmodified action agent \(\pi_A\) interacts with the environment; a separate memory agent \(\pi_M\) is invoked at step 1 and then at a fixed interval (every step in main experiments) with a sliding window of \(k=8\) recent messages plus the current memory bank \(B_t\).

\(\pi_M\) runs two sequential phases: 1. Phase 1 — Memory management: \(\pi_M\) emits structured tool calls (memory_update_status, memory_save_knowledge, memory_save_procedural, memory_delete) to maintain a three-component bank \(B_t = (s_t, K_t, P_t)\): a private status field \(s_t\) (never shown to the action agent), a knowledge store \(K_t\) (stable facts: requirements, env properties, file paths), and a procedural store \(P_t\) (attempts, failures, fixes, diagnostics). 2. Phase 2 — Intervention selection: Conditioned on the updated bank and recent trajectory, \(\pi_M\) either emits a concise targeted reminder \(r_t\) (injected as transient context into the next action-agent call) or the null action \(\emptyset\). The silence option is explicit and encouraged when no memory item is likely to affect the next decision.

Formally: \(B_t \sim \pi_M^{\text{edit}}(\cdot \mid x, w_t, B_{t-1})\); \(i_t \sim \pi_M^{\text{intervene}}(\cdot \mid x, w_t, B_t)\).

For open-weight memory policies, Qwen3.5-27B is first SFT-distilled from prompted memory-agent trajectories (SETA dataset), then fine-tuned with GRPO to calibrate intervention timing.

Key Contributions

  • Formalization of behavioral state decay as the central failure mode of long-horizon LLM agents, distinct from context-length or retrieval coverage issues.
  • A two-phase memory intervention architecture that decouples memory maintenance (what to retain) from intervention selection (when to reactivate), leaving the action agent fully unmodified (plug-and-play).
  • Empirical demonstration that selective proactive injection outperforms passive bank exposure, always-on injection, advisor-only guidance, and Mem0 retrieval baselines.
  • Preliminary evidence that the memory intervention policy is learnable by an open-weight model (Qwen3.5-27B via SFT+GRPO), with partial transfer to held-out Terminal-Bench.

Results

  • Terminal-Bench 2.0 (85 tasks, autonomous command-line execution):
  • Sonnet 4.5: 37.6% → 45.9% (+8.3 pp)
  • Opus 4.6: 43.5% → 45.9% (+2.4 pp)
  • \(\tau^2\)-Bench (278 tasks, interactive tool-use, task-weighted avg):
  • Sonnet 4.5: 55.0% → 61.8% (+6.8 pp); airline +10.0 pp, retail +9.6 pp, telecom +2.6 pp
  • Opus 4.6: 66.2% → 68.7% (+2.5 pp)
  • Ablations on \(\tau^2\)-Bench (Sonnet 4.5):
  • Full memory agent (ours): macro 64.3%, micro 61.2%
  • Full-bank context (no Phase 2 selection): macro 61.5%, micro 60.8%
  • Always inject (no silence option): macro 60.8%
  • Injection-only (no bank): macro 58.6%
  • Mem0 top-10 retrieval: macro 61.5%
  • Baseline: macro 57.5%, micro 55.0%
  • Selective intervention consistently outperforms all ablations; gains hold for stronger action agents, indicating the benefit is not merely compensating for weaker model capacity.

Limitations

  • The memory agent (Claude Opus 4.6) adds non-trivial inference cost at every step; cost analysis is not provided.
  • Fixed-interval triggering (every step) is used to isolate policy effects but is not necessarily optimal; smarter triggers (error-conditioned, context-shift-conditioned) are left as future work.
  • The open-weight training (Qwen3.5-27B SFT+GRPO) achieves only partial transfer to Terminal-Bench; full parity with the prompted Opus memory agent is not demonstrated.
  • Telecom domain shows only +2.6 pp gain for Sonnet 4.5, suggesting domain-specific dynamics that are not fully explained.
  • Evaluation is limited to two benchmarks; generalization to other long-horizon domains (e.g., ML engineering, web navigation) is not empirically validated.
  • The paper text is truncated before the full ablation discussion and the open-weight training section; some details may be incomplete in this summary.

Relevance to Harnesses / Meta-Harnesses

This paper is a direct existence proof for the meta-harness pattern: a separate orchestrating agent that wraps an unmodified action agent and improves its behavior purely through context management, with no changes to the inner agent's weights or prompts. The "proactive memory injection" mechanism — deciding when to surface retained state rather than always prepending it — is exactly the kind of scaffolding logic that belongs in a harness layer, and the plug-and-play design means it can be dropped onto any action agent, making it immediately applicable to harness builders. The behavioral state decay framing also gives a concrete failure mode to test for and guard against in long-horizon agent evaluations, complementing benchmarks like τ²-Bench that already stress multi-turn coherence.