Skip to content

Scaling Agents via Continual Pre-training

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Continual pretraining as scalable approach to improve LLM agent capabilities

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Post-training LLM agents on general-purpose foundation models creates an optimization conflict where models must simultaneously acquire agentic capabilities and align to expert demonstrations. This paper introduces Agentic Continual Pre-Training (Agentic CPT) as an intermediate training stage that instills agentic inductive biases before post-training, producing AgentFounder-30B, which achieves SOTA on 10 deep research benchmarks including 39.9% on BrowseComp-en and 72.8% on GAIA.

Problem

Current open-source deep research agents plateau far below commercial systems (e.g., best open-source at 30.0% vs. OpenAI Deep Research at 51.5% on BrowseComp-en) because SFT and RL are applied directly to general-purpose foundation models that lack agentic inductive biases. This forces post-training to simultaneously learn tool-use behaviors and align them to expert trajectories—creating inherent optimization tensions. Agentic foundation models specifically designed to support downstream fine-tuning remain largely unexplored.

Method

The pipeline inserts Agentic CPT between standard pre-training and post-training using standard next-token prediction \(\mathcal{L} = -\sum_{t=1}^{T} \log P(x_{t+1} | x_1, \ldots, x_t)\) over ~300B synthetic agentic tokens:

Two-stage CPT: - Stage 1 (~200B tokens, 32K context): establishes planning/reasoning skills via ~110B Planning Action tokens, ~30B Reasoning Action tokens, ~40B High-order Action Synthesis (HAS) tokens, plus general-purpose data to mitigate catastrophic forgetting. - Stage 2 (~100B tokens, 128K context): develops long-horizon planning via ~70B long-context HAS tokens (64K–128K sequences) plus 20B replay from Stage 1.

First-order Action Synthesis (FAS) — zero supervisory signal: 1. Knowledge-to-Question Transformation: Unstructured web data is converted into an Entity-Anchored Open-World Knowledge Memory (entity → dense declarative statements), then entity clusters are sampled to synthesize multi-style questions (factual, numerical, multi-hop, survey). 2. Planning Action Synthesis: For each question, \(K\) diverse first-step reasoning+action predictions are generated across stylistically varied question variants (question-level diversity, not sample-level repetition). LLM-as-Judge filters outputs via knowledge alignment verification. 3. Reasoning Action Synthesis: Two-step scheme—(1) model decomposes \(Q\) into sub-questions and speculates preliminary answer \(A_1\) from internal knowledge; (2) given \(Q\) and mapped requisite knowledge, model refines to \(A_2\). No tool calls permitted. Reject sampling via LLM-as-judge on \(A_2\) vs. ground truth.

High-order Action Synthesis (HAS) — with supervisory signal: Given trajectory \(T = \{(S_k, R_k)\}_{k=1}^{K}\) with binary outcome \(J\), for each step \(S_k\) with context \(C_k\): 1. Step-level Scaling: Generate \(N\) alternative "thought and invocation" candidates \(A_k^{(1)}, \ldots, A_k^{(N)}\) without executing tools; merge with original \(S_k^{(0)}\) and shuffle. 2. Contrastive Decision-Action Synthesis: Reformulate the trajectory as an explicit multi-option decision process: enumerate all \(N+1\) options, insert "I will choose option \(n_k\)," append real tool response \(R_k\), and conclude with judgment text "My decision is {Correct/Incorrect}" (from \(J\)). This reuses suboptimal/discarded trajectories as rich training signal without requiring step-level reward models.

Key Contributions

  • First systematic incorporation of Agentic CPT as an intermediate training stage for building agentic foundation models.
  • FAS: scalable zero-supervision agentic data synthesis via entity-anchored knowledge memory and multi-style question generation (~200B tokens without API trajectory costs).
  • HAS: step-level scaling that converts suboptimal trajectories into contrastive decision-making training data, bypassing uncertain step-level reward assignment.
  • AgentFounder-30B: new SOTA among open-source agents on 10 benchmarks under the single-agent ReACT paradigm.
  • Empirical demonstration of scaling law behavior—steady performance gains as Agentic CPT data volume increases.
  • Adaptability: AgentFounder-Base outperforms Qwen3-30B-A3B-Base under identical SFT data across all configurations, validating CPT's downstream transferability.

Results

Evaluated under single-agent ReACT; AgentFounder-30B vs. best open-source baselines:

  • BrowseComp-en: 39.9% vs. DeepSeek-V3.1 at 30.0% (+9.9pp); approaches OpenAI-o3 (51.5%) and Deep Research (51.5%).
  • BrowseComp-zh: 43.3% vs. GLM-4.5 at 37.5% (+5.8pp); behind DeepSeek-V3.1 (49.2%) and OpenAI-o3 (58.1%).
  • GAIA: 72.8% — highest single-agent accuracy reported; exceeds OpenAI-o3 (70.5%).
  • xbench-DeepSearch: 73.0% — highest reported.
  • HLE Pass@1: 31.5% vs. DeepSeek-V3.1 (29.8%) and OpenAI Deep Research (26.6%).
  • Frames Pass@1: 89.6% — highest reported, surpassing Deep Research (83.7%).
  • SEAL-0 Pass@1: 43.9% vs. DeepDive-32B (42.6%).
  • DeepResearch Bench RACE Overall: 47.9% vs. Qwen3-235B-A22B (44.8%).
  • Adaptability ablation (same SFT-B data): AgentFounder-30B-Base vs. Qwen3-30B-A3B-Base: +11.3pp BrowseComp-en, +7.7pp BrowseComp-zh, +1.0pp GAIA, +4.5pp HLE.

Limitations

  • Underperforms DeepSeek-V3.1 and OpenAI-o3 on BrowseComp-zh; attributed to limited Chinese training data and suboptimal Google Search performance in Chinese.
  • Evaluation restricted to the single-agent ReACT paradigm; multi-agent or other scaffolding results not reported.
  • FAS planning action synthesis generates only first-step reasoning without end-to-end trajectory execution; full trajectory quality depends on post-training.
  • HAS relies on binary trajectory-level judgment \(J\) to propagate credit; intra-trajectory step correctness is not independently verified.
  • Compute costs of ~300B CPT tokens (on top of base pre-training) are substantial and not fully analyzed.

Relevance to Agentic AI / LLM Agents

This paper directly attacks the bottleneck of building capable open-source agents by reframing the problem as a foundation-model quality issue rather than a post-training recipe issue, establishing a new training paradigm (Pre-train → Agentic CPT → Post-train) that separates capability acquisition from behavioral alignment. The FAS and HAS synthesis techniques provide scalable recipes for generating agentic training data without expensive live API trajectory collection, which is a significant practical contribution to the field. The demonstrated scaling law behavior suggests that agentic capability can be systematically scaled through data, analogous to general LLM pre-training, opening a new axis of scaling for agent research. For practitioners building open-source deep research or tool-use agents, this work establishes that investing in an intermediate CPT stage yields consistent downstream gains across diverse post-training configurations.