Skip to content

Shared Selective Persistent Memory for Agentic LLM Systems

🕒 Published (v1): 2026-07-10 15:07 UTC · Source: Arxiv · link

Why this paper was selected

Shared selective persistent memory solves session-zero cold-start in multi-turn tool-using agents; immediately buildable

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces shared selective persistent memory, a memory architecture for agentic LLM systems that persists four categories of reusable context (task specifications, data schemas, tool configurations, output constraints) across sessions while explicitly discarding session traces. A deployed enterprise platform validates the approach, achieving 96% task completion vs. 79% (no memory) and 71% (full history), with a complementary zero-token data refresh mechanism that eliminates LLM re-invocation for recurring data updates entirely.

Problem

Agentic LLM systems are stateless: each session discards accumulated domain constraints, data schemas, tool configurations, and output format preferences, forcing users to re-specify context on every invocation. The naive fix—injecting entire conversation histories—is counterproductive: prior tool-use traces bias the agent toward stale solution paths and inflate token costs, reducing task completion by 8 percentage points relative to no memory at all.

Method

The architecture decomposes reusable session knowledge into four orthogonal memory categories:

  1. \(M_{\text{task}}\) — custom system prompt extensions encoding domain rules and quality constraints
  2. \(M_{\text{data}}\) — precomputed statistical profiles (column types, distributions, categorical catalogs, sample rows) generated via pandas.describe() extensions, reducing raw data to ~500 tokens
  3. \(M_{\text{tools}}\) — available connectors with parameter schemas, invocation patterns, and auth requirements
  4. \(M_{\text{output}}\) — structural contracts between generated artifacts and the runtime environment (e.g., data-injection contract)

At session start, the system prompt is composed as: $\(s' = s_{\text{base}} \oplus M_{\text{task}} \oplus M_{\text{tools}} \oplus M_{\text{output}}\)$ and the data schema injected into the user message: $\(q' = q \oplus M_{\text{data}}\)$

Session traces (reasoning chains, tool invocation logs, error-recovery paths, intermediate files) are explicitly discarded. A zero-token data refresh mechanism enforces a runtime data-injection contract: generated programs consume data from a parameterized injection point, never hardcoded values. On new data arrival, schema compatibility is checked; if compatible (original columns ⊆ new columns), the artifact re-renders with zero LLM tokens. Workspaces are shared via role-based access control (OIDC, three roles: owner/steward/viewer), with git-backed versioning and draft isolation. The platform uses Claude Opus 4 for artifact generation and Claude Sonnet 4 for lightweight subtasks.

Key Contributions

  • Four-category selective memory decomposition for agentic LLM sessions, with formal motivation for what to discard
  • Zero-token data refresh mechanism via runtime data-injection contracts, enabling artifact reuse without LLM re-invocation
  • Collaborative workspace sharing with RBAC, git-versioned artifacts, and draft isolation
  • Empirical demonstration that full-history persistence is strictly worse than no memory (71% vs. 79% completion), establishing selective forgetting as necessary rather than optional
  • Deployed enterprise system evaluation across three use-case families and replication on four public datasets

Results

  • Task completion (24 enterprise tasks): Selective memory 96% vs. no memory 79% vs. full history 71%
  • User turns: Selective 1.4 vs. no memory 4.3 vs. full history 3.1
  • Wall-clock time: 68s (selective) vs. 285s (no memory) vs. 310s (full history)
  • Input tokens: 3.4K (selective) vs. 2.1K (no memory) vs. 18.7K (full history)
  • Statistical significance: selective vs. no memory \(p=0.046\); selective vs. full history \(p=0.008\) (Fisher's exact)
  • Zero-token refresh: 12/12 successful trials on public datasets; 18/24 on enterprise data (schema-compatible subset)
  • Token efficiency (summary vs. raw injection): 97× reduction on enterprise data (0.5K vs. 48.7K tokens); 946× on public datasets (0.5K vs. 473K tokens)
  • User study (N=12): Recurring generation 14× faster (12s vs. 165s); refinement 2.5× faster; "would use again" Likert 6.5 vs. 4.2/7
  • Public dataset replication (4 datasets, 36 trials): Selective memory 100% completion; full history 75% (−8pp vs. no memory's 83%)

Limitations

  • Evaluation corpus is small (24 enterprise tasks, 12 user-study participants), limiting statistical power
  • Memory decomposition was designed for artifact-generation workspaces; applicability to open-ended agentic tasks (e.g., web browsing, code repair) is untested
  • Schema compatibility check is structural (column subset); semantic drift (same column name, changed meaning) is not detected
  • The single selective-memory failure involved cross-file join semantics missed by schema summaries, indicating the data profile abstraction has blind spots for relational reasoning
  • User study participants limited to engineers and analysts; generalizability to less technical users unclear
  • No evaluation of memory staleness or update cadence when \(M_{\text{task}}\) or \(M_{\text{tools}}\) evolve over time

Relevance to Agentic AI / LLM Agents

This work directly addresses the session-statefulness gap in agentic systems, a fundamental architectural weakness present in ReAct, LangChain, AutoGen, and most deployed agent frameworks. The key finding—that naive full-history context injection actively degrades performance relative to no memory, due to trace anchoring—is an important empirical caution for agent system designers who treat more context as strictly better. The four-category decomposition (specifications, schemas, tools, output contracts) offers a principled taxonomy for what "persistent agent state" should contain, distinct from RAG, summarization, or skill libraries like Voyager. The zero-token refresh mechanism also exemplifies a design pattern of architectural separation between LLM-generated programs and runtime data, which generalizes to any agentic setting involving recurring structured-data tasks.