Skip to content

AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents

🕒 Published (v1): 2026-07-02 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgenticSTS introduces a bounded-memory contract for long-horizon LLM agents: instead of appending raw transcripts, each decision prompt is freshly assembled from five typed, ablatable layers. Instantiated on Slay the Spire 2—a stochastic deck-building game where frontier LLMs currently win 0% at the easiest difficulty—the testbed shows the largest observed gain comes from enabling triggered strategic skills (L5: 3/10 → 6/10 wins), though this difference is directional rather than statistically significant at N=10.

Problem

Long-horizon LLM agents typically accumulate context by appending all prior observations, tool calls, and reflections to each prompt. This approach (a) grows the prompt with \(O(d \cdot \bar{s})\) complexity for \(d\) decisions, (b) obscures which memory component drives performance, and (c) prevents controlled ablation. No existing benchmark simultaneously enforces a bounded per-decision context budget and provides layer-level ablation on a hard, unsaturated long-horizon task.

Method

The agent, AgenticSTS, implements a per-decision compositional context interface. For decision \(d\) at state \(s_d\), the prompt is assembled as:

\[u_d = \pi\!\left(L_1, L_2(s_d), L_3(s_d), L_4(s_d), L_5(s_d)\right)\]

The five typed layers differ in mutability: - L1: immutable role/protocol instructions - L2: state-typed schemas and legal action formats - L3: retrieved game-rule records (cards, relics, enemies; refreshed per patch) - L4: postrun episodic summaries, keyed by character × ascension × act × enemy class - L5: triggered strategic skill guides, each with an explicit trigger condition, prose policy, and a four-level write gate (cosine → Jaccard → LLM judge → optional reap)

No raw cross-decision transcript is appended; the prompt size is \(O(|sys| + s_\text{thread} + \sum_i k_i \cdot s_i)\), constant with respect to run length. Skills are populated either via human-authored seed bodies (Mode A) or stub-template-filled self-evolving distillation (Mode B). Combat prompts are capped to at most three messages per round; a dispatcher routes calls to four model tiers (fast/strategic/analysis/evolution), yielding a median of 67 strategic LLM calls per run rather than one per game action.

The evaluation uses a fixed-A0 five-cell ablation (N=10 per cell), a cross-backbone probe (Qwen 3.6 27B, DeepSeek V4 Pro, Gemini 3.1 Pro), and an auto-mode ascension ladder that allows postrun memory writes.

Key Contributions

  • A formal bounded-memory contract expressed as five typed, individually ablatable layers, separating prompt construction from raw history accumulation
  • Evidence that triggered L5 strategic skills produce the largest observed A0 win-rate difference in the balanced matrix (3/10 → 6/10), with the gap being directional (Fisher exact \(p \approx 0.37\)) at the current sample size
  • Mode B self-evolving template skills match human-authored Mode A skills at A0 (score-diff CI \([-18.6, +24.8]\)), separating skill-layer existence from prose source
  • Cross-backbone probe showing the Gemini-trained frozen stack transfers to Gemini but degrades substantially on Qwen and DeepSeek, indicating backbone sensitivity
  • A public archive of 298 complete trajectories with condition tags, SHA-anchored L4/L5 snapshots, per-decision prompt records, and Wilson/bootstrap analysis scripts for community reuse

Results

  • Baseline-strict (no scaffold): 3/10 A0 wins (Wilson 95% CI [10.8%, 60.3%]); mean score 70.4
  • Prompt-only (no L4/L5): 4/10 wins; \(\Delta_\text{prompt} = +1/10\)
  • All L5-scaffolded cells (mode-a, mode-b-frozen, full-frozen): 6/10 each; \(\Delta_{L5} = +2/10\) over no-scaffold
  • Fisher exact on 3/10 vs. 6/10: \(p \approx 0.37\); pooled scaffolded vs. unscaffolded (18/30 vs. 7/20): \(p \approx 0.148\)—differences are directional, not statistically significant
  • Auto-mode ladder: postrun-active streams (Mode B self-evolve, full+postrun) reach A6–A8; frozen/no-postrun streams plateau at A2–A4
  • Cross-backbone: Qwen and DeepSeek win 0/5 in both baseline-strict and full-frozen cells; Gemini full-frozen score CI [63.1, 96.5]; the frozen stack is strongly backbone-specific
  • Token audit: bounded contract maintains \(O(1)\) per-decision token cost vs. a transcript-appending counterfactual at \(\frac{1}{4}O(c^2)\) growth

Limitations

  • No matched accumulating-context control condition within the same codebase; whether the bounded contract outperforms a raw-transcript agent is explicitly left as future work
  • N=10 per cell is insufficient for statistical significance; all fixed-A0 win-rate comparisons are directional only
  • The frozen L4/L5 stack was derived from Gemini trajectories; cross-backbone transfer to Qwen and DeepSeek fails, limiting generality claims
  • The auto-mode ladder reports endpoint reach, not win rates at each difficulty level, so upper-difficulty performance is incompletely characterized
  • Results are specific to one game; generalization to other long-horizon environments is unvalidated
  • The 52/3 scoring coefficient is heuristic; score-based comparisons are checked under ±10% perturbation but remain sensitive to this choice

Relevance to Agentic AI / LLM Agents

This work directly addresses the context/memory policy problem at the core of long-horizon agent design: it formalizes "what the agent is allowed to see per decision" as an explicit, ablatable contract rather than an engineering default. For researchers building or evaluating agents with tool use, multi-step planning, or episodic memory (e.g., Voyager-style skill libraries, MemGPT-style stores), AgenticSTS provides both a methodology—typed retrieval with independent layer toggling—and a reproducible 298-trajectory dataset for studying context use. The finding that triggered strategic skills (L5) produce the largest observed lift, while episodic summaries (L4) show no separable A0 effect, is a concrete, if tentative, signal about which memory types matter in stochastic decision-heavy environments. The testbed's unsaturated difficulty regime (0% frontier-model wins at easiest tier) also makes it a credible long-horizon benchmark for future agent designs.