Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory

🕒 Published (v1): 2026-07-20 04:43 UTC · Source: Arxiv · link

Why this paper was selected

Budget-aware retain-vs-consolidate memory operator; directly addresses context-cost tradeoffs in agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper formalizes the decision of when a language agent should compress its memory (consolidation) versus keep raw records (retention) as a function of token budget pressure. It derives a two-term utility decomposition and implements a lightweight offline learner (OAS) that selects among Retain, Merge, Abstract, and Rewrite before generation. On LongMemEval, consolidation improves absolute accuracy by up to 48% under tight budgets while retention wins by 8–11% under loose budgets.

Problem

Language agent memory systems face a hard token-budget constraint from LLM context windows and inference costs. Two dominant strategies—retention (keep raw records) and consolidation (compress records)—are each optimal only in specific regimes, but no prior work formalizes when to consolidate or which of the generative operators (Merge, Abstract, Rewrite) to apply. The decision is budget-dependent yet existing systems either always retain or always consolidate without adaptive selection.

Method

The paper formalizes a one-step query-time decision over four actions \(A = \{\text{ret}, \text{Merge}, \text{Abstract}, \text{Rewrite}\}\). For a fixed query \(q\), candidate cluster \(C \subseteq M\), and budget \(B\), each action has a conditional expected utility \(\mu_a(z)\); the optimal policy selects \(\arg\max_{a \in A} \mu_a(z)\), or retains on tie.

To explain budget dependence, Proposition 1 decomposes each generative operator's surrogate utility gain as: $\(\Delta U_{B,o} = \underbrace{C_{B,o}}_{\text{coverage effect}} + \underbrace{R_{B,o}}_{\text{replacement effect}}\)$ where \(C_{B,o}\) is the gain from covering queries missed by retention (positive when the generated record fills gaps), and \(R_{B,o}\) is the signed change for queries already answered by raw records (can be negative if compression loses fidelity). As budget \(B\) decreases, the raw-grounded region \(G_B\) shrinks, increasing the coverage effect's dominance and making consolidation more attractive.

Offline Abstraction-Safety (OAS) operationalizes this as a lightweight router: it fits four ridge regression utility functions \(\hat{\mu}_a(z) = b_a + \langle \theta_a, \psi(z) \rangle\) on an 11-dimensional pre-generation feature vector \(\psi(z)\) (normalized budget \(B/512\), token pressure, budget-fit fraction \(f_B\), constituent count, session count, embedding cluster inconsistency, mean pairwise cosine similarity, 4-way query-type one-hot). Supervision is offline paired evaluation of all four actions. A calibrated variant adds a per-budget harm threshold \(\tau_{\text{safe},B}\) fitted on held-out validation to suppress harmful consolidations (those that lower accuracy vs. retention).

Key Contributions

• Unified when–which formulation: Places retention and three generative operators in a single budget-dependent decision space with signed per-operator utility.
• Testable mechanism (Propositions 1–2): Decomposes consolidation value into a coverage effect (positive, grows under tight budgets) and a replacement effect (signed, penalizes fidelity loss), predicting a sign reversal at a crossover budget.
• OAS learner: Lightweight multi-action ridge-regression router operating on pre-generation features with held-out harm calibration; an MLP variant is evaluated as a capacity-control baseline.
• Empirical validation of crossover: LongMemEval and LoCoMo both exhibit the predicted budget-dependent sign reversal; the crossover occurs at a smaller absolute budget on LoCoMo (consistent with its shorter evidence, supporting relative rather than absolute threshold).

Results

• LongMemEval, tight budget: Consolidation improves absolute accuracy by up to 48% over retention.
• LongMemEval, loose budget: Retention outperforms every consolidation operator by 8–11%.
• A second answer model reproduces the reversal, confirming generality.
• LoCoMo: Same crossover pattern at a smaller absolute budget (shorter evidence → pressure shifts earlier), supporting relative budget pressure as the governing quantity.
• Operator ranking: Cross-note Abstract and Merge generally outperform local Rewrite when compression is necessary; neither cross-note operator universally dominates the other.
• OAS: Improves the controlled when–which decision under budget-stressed conditions; the full-history variant does not surpass the strongest fixed policy, bounding the claim to budget-aware selection rather than general-purpose instance routing.
• MLP does not reliably improve over ridge regression, confirming that the action-utility relationship is low-dimensional.

Limitations

• OAS is evaluated in a controlled oracle-evidence protocol; in end-to-end deployment, the candidate-evidence cost and query type must be estimated by a retriever and type classifier.
• The full-history OAS variant does not surpass the best fixed policy, so instance-level routing beyond the controlled setting is not yet demonstrated.
• The idealized mechanism model (Proposition 1) relies on a localized retrieval change assumption (Assumption 1) that excludes retrieval spillovers outside \(H_B\); real retrievers may violate this.
• Only one query–cluster decision is analyzed at a time; interactions among repeated memory updates and background consolidation are explicitly out of scope.
• The coverage and replacement effects' latent fidelity functions (\(\phi_m(q)\)) are not identified; the model explains direction, not magnitude.
• Experiments are limited to two benchmarks (LongMemEval, LoCoMo) with specific evidence-length distributions.

Relevance to Agentic AI / LLM Agents

Memory management under token-budget constraints is a foundational problem for long-horizon LLM agents that accumulate observations, dialogue, and retrieved facts across many turns. This work provides the first principled framework for adaptive selection between retention and generative compression, directly addressing a gap in agent memory systems (e.g., MemGPT, Mem0, A-MEM, RecMem). The budget-dependent crossover finding is immediately actionable for system designers: under-resourced deployments should consolidate aggressively, while memory-abundant agents should prefer raw retention to avoid fidelity loss. The lightweight OAS design—11 features, ridge regression, offline training—is practical for real agent pipelines.