Skip to content

Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents

🕒 Published (v1): 2026-07-14 02:10 UTC · Source: Arxiv · link

Why this paper was selected

Epistemic working memory structure explicitly tracks confirmed/open sub-goals; scales multi-hop agent reasoning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SLEUTH is a prompting-only framework that gives language agents an explicit tripartite working memory—Confirmed Facts, Active Hypotheses, and Open Questions—to combat context dilution in multi-hop reasoning. It outperforms five established baselines across five benchmarks, with gains scaling from +4.7 EM points on 2-hop to +10.9 on 4-hop chains. A lightweight commitment trigger resolves a novel failure mode (over-verification), but only when the agent already maintains structured state.

Problem

Multi-hop QA agents degrade sharply as reasoning chains lengthen—not because individual steps are hard, but because investigative state (what is confirmed, suspected, or unknown) lives only implicitly in a growing context window. Early discoveries are progressively buried under later retrievals ("context dilution"), causing errors in intermediate state tracking and budget exhaustion without commitment.

Method

SLEUTH augments any instruction-following agent via a system prompt only—no fine-tuning, no architecture changes, no extra episodes. At every turn \(t\), the agent maintains and outputs a working memory state \(W_t = (F_t, H_t, Q_t)\):

  • Confirmed Facts \(F_t\): verifiable statements grounded to source titles, monotonically accumulated (\(F_t \supseteq F_{t-1}\)).
  • Active Hypotheses \(H_t\): candidate answers ranked by evidentiary confidence (high/medium/low), each linked to supporting/contradicting fact IDs.
  • Open Questions \(Q_t\): outstanding uncertainties, each with a proposed action and priority; the highest-priority question directly drives the next tool call.

After each tool call, the agent (1) extracts new facts, (2) updates hypothesis confidence, (3) prunes answered questions and generates new ones, and (4) selects the next action from \(Q_t\).

A nudge (commitment trigger) fires when \(t \geq \alpha T\) (turn budget fraction, default \(\alpha=0.7\)) and \(|F_t| \geq \tau\) (minimum facts, default \(\tau=2\)), instructing the agent to synthesize an answer from accumulated evidence rather than continue searching. For weaker models, a runtime check rejects malformed responses and re-prompts until the structure is present, raising adherence from ~15% to ~99%.

Key Contributions

  • SLEUTH framework: tripartite epistemic working memory implemented via prompting alone; Confirmed Facts monotonicity prevents context dilution; Open Questions directly ground action selection.
  • Difficulty-scaling gains: EM improvement over the strongest baseline grows from +4.7 (HotpotQA, 2-hop) to +10.9 (MuSiQue, 4-hop); consistent on both Claude Sonnet 4.6 and GLM-5.
  • Evidence sufficiency problem: identifies over-verification as a distinct failure mode—92% of timed-out episodes already contain the gold answer in history; the nudge mechanism converts latent correct reasoning into committed answers.
  • Nudge requires structured state: applying the identical commitment trigger to an unstructured ReAct agent yields +0.2 EM (noise); SLEUTH gains +4.0 EM, isolating structured memory as the necessary condition.
  • Protocol adherence finding: enforcing working-memory maintenance on GLM-5 recovers up to +19.0 EM points on 4-hop, demonstrating that reasoning organization outweighs raw model capability.

Results

All experiments use 20-turn budget, top-\(k\)=3 retrieval, Exact Match metric.

  • HotpotQA (2-hop): SLEUTH 66.1% vs. Reflexion 61.4% (+4.7); all gaps \(p<0.001\) by paired bootstrap.
  • MuSiQue 2-hop: SLEUTH 66.5% vs. Reflexion 54.6% (+11.9).
  • MuSiQue 3-hop: SLEUTH 63.7% vs. Reflexion 53.3% (+10.4).
  • MuSiQue 4-hop: SLEUTH 53.1% vs. Reflexion 42.2% (+10.9); nudge lifts from 49.1% (no nudge) to 53.1%.
  • 2WikiMultiHopQA: SLEUTH 76.4% vs. Reflexion 73.6% (+2.8).
  • Commitment ablation (4-hop): no nudge → 24.7% timeout rate, 64.6% conditional accuracy; Nudge@14 (α=0.7) → 11.1% timeout, 58.9% conditional accuracy; EM peaks at Nudge@14 (53.1%).
  • GLM-5 cross-model: gains +1.0 to +11.1 EM; protocol enforcement alone (no architectural change) recovers +8.9 / +11.7 / +19.0 on 2/3/4-hop MuSiQue.
  • Ablations (4-hop): Notes (unstructured scratchpad) 41.7%; Facts-Only 49.9%; Facts+Hyp. 48.9%; Facts+Q. 47.7%; full SLEUTH 53.1%—tripartite structure adds +3.2 over best two-component variant.
  • Error decomposition (4-hop): 32.1% reasoning errors, 11.1% timeouts, 4.4% verbose answers; inference quality over retrieved evidence is the primary bottleneck.

Limitations

  • Retrieval oracle assumption: all benchmarks provide the full paragraph context to the retrieval toolkit; open-domain retrieval performance is not evaluated.
  • Single-trajectory scope: SLEUTH does not address multi-episode memory or cross-question knowledge transfer, which Reflexion, CLIN, and ExpeL target.
  • Commitment trigger is heuristic: \(\alpha=0.7\) and \(\tau=2\) are empirically selected; generalization to different turn budgets or domains is not characterized.
  • Structured prompt overhead: the mandatory three-section output on every turn increases token consumption; cost impact is not reported.
  • Primary bottleneck unresolved: 32.1% of remaining errors are inference errors over correctly retrieved evidence—SLEUTH does not address reasoning quality, only state organization.
  • Model scope: only two model families (Sonnet 4.6, GLM-5) tested; behavior on smaller or open-source models with weaker instruction following is extrapolated but not fully characterized.

Relevance to Agentic AI / LLM Agents

SLEUTH directly addresses one of the most persistent failure modes of tool-using agents—context dilution under long reasoning chains—and does so at the lightest design point (prompting alone), making it immediately applicable to any deployed LLM agent. The finding that structured epistemic state is a necessary condition for the commitment trigger to work has architectural implications: it suggests that agent scaffolds should treat working memory as a first-class citizen, not an optional add-on. The protocol adherence experiment provides a strong empirical argument that agent framework design (how state is organized and enforced) can outweigh raw model capability, directly relevant to work on agent scaffolding, multi-hop RAG pipelines, and reasoning-action loops. The identification of the evidence sufficiency problem as a distinct failure mode (separate from retrieval quality and reasoning quality) adds a new diagnostic category to the multi-hop agent literature.