Skip to content

When Does Continual Learning Require Learning

🕒 Published (v1): 2026-07-08 18:27 UTC · Source: Arxiv · link

Why this paper was selected

Challenges framing of continual learning for LLMs — argues beyond context management, relevant to long-lived agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper argues that continual learning for LLMs is not a single capability but a family of problems requiring fundamentally different update strategies. The authors introduce a mechanism-agnostic protocol and evaluate eight methods across four families on tasks spanning domain shift, discrete fact updates, temporal drift, and agentic state accumulation. No single method wins across all regimes; the right update mechanism is determined by the nature of environmental change.

Problem

The field has conflated distinct forms of non-stationarity under the single label "continual learning," defaulting to forgetting-prevention as the primary objective. This framing misses a complementary failure mode — catastrophic memorizing, the inability to overwrite outdated beliefs — and ignores agentic settings where the model's own actions cause environmental drift. Existing evaluations are also method-coupled, making prompt-based, weight-update, and architectural approaches incomparable on equal footing.

Method

The authors define a unified protocol: a learner sees stages \(T_1, \ldots, T_K\) in fixed order; at each stage it applies an update operator \(\theta_k = U_k(\theta_{k-1}, D_k^{tr})\) where \(U_k\) is unrestricted (prompt edit, full fine-tune, adapter, external memory write, or no-op). After every stage, all eval sets are scored, yielding a forgetting matrix \(R_{i,j} = \text{acc}(\theta_i, D_j^{eval})\). Two aggregate signals are reported: $\(\text{BWT} = \frac{1}{K-1}\sum_{k=1}^{K-1}(R_{K,k} - R_{k,k}), \qquad \text{FWT} = \frac{1}{K-1}\sum_{k=2}^{K}(R_{k-1,k} - R_{0,k})\)$ Eight methods on Qwen3-8B (non-thinking) are evaluated under this shared protocol across four task settings: (1) domain transfer (ToolUse → FinQA → SciKE-Bio); (2) selective fact update (TempWiki — monthly Wikipedia diffs, post-cutoff Nov 2025–Feb 2026); (3) noisy temporal drift (10-K SEC filings, 2015–2020, stock direction prediction); (4) agentic accumulation (agent-generated stage ordering where prior actions determine future task state).

Key Contributions

  • A formal definition of continual learning as increasing competence as the world changes, decomposed along spatial (domain shift) and temporal (fact drift, gradual drift, agentic accumulation) axes.
  • A mechanism-agnostic sequential protocol enabling fair comparison of prompt, weight-update, and architectural methods under identical compute budgets.
  • The concept of catastrophic memorizing — the symmetric failure to updating stale beliefs — as a necessary counterpart to catastrophic forgetting.
  • A benchmark suite spanning domain adaptation, post-cutoff Wikipedia fact updates, financial temporal drift, and agentic accumulation tasks.
  • A systematic empirical characterization of eight methods (GEPA, ACE, SFT, SDFT, GRPO, SDPO, Cartridges, In-place TTT) across these regimes.

Results

  • Domain transfer: SDFT is the only method whose three final-stage scores are all at or above zero-shot baselines; GEPA achieves ~40% gain on FinQA but nearly fully regresses after the next stage; Cartridges and In-place TTT show flat or degrading performance across stages.
  • Fact update (TempWiki): GRPO is the only weight-update method whose stable-fact F1 does not move opposite to drift F1; it achieves the largest measured gain at \(s_2\) (+6.2 points). SDFT's stable-fact F1 falls from ~0.30 to ~0.15 over three slices. GEPA achieves highest drift F1 (0.24 at \(s_3\)) but at cost of stable F1 degradation. Cartridges averages \(-0.8\) on drift with no consistent improvement over baseline.
  • Noisy temporal drift (10-K): Distillation-based methods (SDFT, SDPO, Cartridges) sustain forward accuracy; GRPO degrades significantly when reward signal is noisy; SFT stays near chance throughout; prompt methods (GEPA, ACE) degrade on future years.
  • Agentic accumulation: Both ACE (prompt-based playbook) and SFT (weight update) improve over zero-shot at every chain length, though absolute success decays as chain length grows.

Limitations

  • Experiments use a single backbone (Qwen3-8B non-thinking); results may not generalize to other scales or architectures.
  • The per-stage compute budget \(C\) is held fixed across methods but not cross-validated for optimality; some methods may be systematically under- or over-resourced.
  • TempWiki evaluation uses binary scoring for many relation types, introducing noise acknowledged by the need for a filtered "TempWiki-Easy" variant in the appendix.
  • The agentic accumulation setting is described but results are limited to chain-length decay trends; no fine-grained breakdown by action type or environment.
  • No oracle upper bound (e.g., full joint training on all stages) is provided as a ceiling reference.
  • The 10-K financial task has an inherently weak and noisy label signal, making it difficult to isolate method failure from task-level irreducible uncertainty.

Relevance to Agentic AI / LLM Agents

The agentic accumulation axis directly models a core challenge in deployed LLM agents: the agent's own actions reshape the environment, so each episode's task distribution is endogenous to the agent's prior behavior — the LLM analogue of the frame problem. The finding that both prompt-based (ACE) and weight-based (SFT) methods improve with chain length but degrade in absolute success as chains grow has direct implications for long-horizon agent design, suggesting that neither external scaffolding nor parameter updates alone are sufficient for reliable agentic operation over extended interaction sequences. The protocol itself is a valuable contribution for agent evaluation: by casting sequential experience as a forgetting matrix, it provides a rigorous way to measure whether an agent's accumulated experience actually transfers forward to novel situations rather than merely fitting past trajectories.