AREX: Towards a Recursively Self-Improving Agent for Deep Research¶

🕒 Published (v1): 2026-07-23 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

AREX is a family of Recursively Self-Improving (RSI) deep research agents that exploit the discovery–verification asymmetry in multi-constraint research tasks: rather than searching longer, AREX alternates between an inner evidence-gathering loop and an outer constraint-wise audit loop that converts partially verified answers into better-targeted follow-up problems. Trained on verified synthetic tasks via multi-stage agentic mid-training and long-horizon RL with key-step emphasis, AREX-Base (122B-A10B MoE) and AREX-Turbo (4B dense) outperform comparable-scale baselines across deep-search, wide-search, and reasoning benchmarks.

Problem¶

Existing deep research agents address multi-constraint queries by extending a single search trajectory with more compute, but this does not guarantee systematic progress: early errors persist, exhausted directions are revisited, and partially valid candidates are accepted prematurely. The core issue is that discovering a jointly satisfying answer is expensive, yet verifying individual constraints on a candidate is tractable — existing systems do not exploit this asymmetry to adaptively refocus search on remaining unresolved claims. Additionally, long-horizon research causes interaction histories to balloon with redundant, contradictory, and obsolete content, and fixed heuristics for context management (token-threshold truncation, rule-based discarding) destroy decision-relevant information such as negative evidence and source provenance.

Method¶

AREX implements a hierarchical bi-level recursive loop:

Inner research loop — given research objective $q^{(k)}$, the policy $\pi_\theta$ iteratively produces $(m^{(k)}_t, a^{(k)}_t, o^{(k)}_t)$ triples (analysis, action, observation). At any point the model may invoke a learned update_context tool that compresses the accumulated trajectory $h^{(k)}_t$ into a compact improvement state $z^{(k)}_t$ preserving verified findings, current candidates, unresolved constraints, rejected candidates, and the next-step plan; subsequent actions condition only on the refreshed effective context $\bar{h}^{(k)}_t = z^{(k)}_\tau \oplus [m,a,o]_{\tau+1}^t$. The loop terminates via a finish call that externalizes a provisional answer $y^{(k)}$, evidence set $\mathcal{E}^{(k)}$, and a confidence score $s^{(k)} \in [0,100]$.

Outer self-improvement loop — applies the decision rule: $$d^{(k)} = \begin{cases} \text{Accept} & s^{(k)} \ge \tau \\ \text{Refine} & s^{(k)} < \tau \wedge v^{(k)} = 1 \\ \text{Restart} & s^{(k)} < \tau \wedge v^{(k)} = 0 \end{cases}$$ where $v^{(k)} \in \{0,1\}$ is a recoverability flag. On Refine, preserved progress $P^{(k)}$ and identified issues $I^{(k)}$ seed round $k{+}1$; on Restart, the model re-initializes from the original query $x$.

Training proceeds in three stages: (1) browse-intensive multi-round SFT to install tool-use; (2) expert reasoning SFT to strengthen long-form deduction; (3) mixed-capability consolidation with selective trajectory replay concentrated on key steps (decisive evidence acquisition, contradiction resolution, direction correction) to address sparse credit assignment. A final long-horizon RL stage optimizes for verified task outcomes.

Backbone models: Qwen3.5-4B (Turbo) and Qwen3.5-122B-A10B (Base).

Key Contributions¶

Formal framing of multi-constraint deep research as a Recursively Self-Improving process driven by the discovery–verification asymmetry.
Bi-level AREX framework with constraint-wise outer loop governing Accept/Refine/Restart transitions and evidence-aware termination.
A learned update_context tool invoked autonomously by the agent itself, performing research-state consolidation rather than generic summarization, without a separate external model.
Multi-stage training recipe: progressive agentic mid-training followed by long-horizon RL with key-step focused credit assignment on synthesized verifiable tasks.
Instantiation at two scales (4B dense, 122B-A10B MoE) with consistent improvements over comparable-scale baselines.

Results¶

BrowseComp: AREX-Base 82.5, AREX-Turbo 70.7 (vs. Kimi-K2.6 83.2; AREX-Base competitive with top closed models)
GAIA (Closed / Open): AREX-Base 85.4 / 71.0, AREX-Turbo 81.6 / 57.0 (vs. Miro-H1 88.5 closed; AREX-Base beats Kimi-K2.6 80.6 closed)
DeepSearchQA: AREX-Base 82.0, AREX-Turbo 68.5 (vs. Kimi-K2.6 80.8, GPT-5.4 77.5)
WideSearch: AREX-Base 52.4, AREX-Turbo 40.6 (vs. GLM-5 50.4; AREX-Base leads this benchmark)
HLE (with Tools): AREX-Base 89.9, AREX-Turbo 78.5 (vs. DeepSeek-Pro 88.7, GPT-5.4 88.5; AREX-Base tops the leaderboard shown)
AREX-Base (10B activated) remains competitive with models using substantially more activated parameters across all benchmarks.

Limitations¶

The confidence threshold $\tau$ and maximum number of recursive rounds are predefined hyperparameters with no self-calibration mechanism described.
Restart discards the entire trajectory; if the model's recoverability estimate $v^{(k)}$ is miscalibrated, useful partial evidence is irretrievably lost.
Training data construction requires strong teacher models to generate high-quality trajectories, making the pipeline expensive and dependent on the availability of capable oracle models.
Evaluation is on English-centric benchmarks; cross-lingual and domain-specific research settings are not addressed in the excerpted text.
The paper text is truncated before the full ablation studies and explicit limitations section, so additional known limitations may not be represented here.

Relevance to Agentic AI / LLM Agents¶

AREX directly advances the core architecture problem in long-horizon LLM agents: how to maintain productive research state over many steps without context explosion or premature commitment to incorrect hypotheses. The bi-level RSI loop (inner gather/verify, outer diagnose/redirect) is a concrete, trainable instantiation of the general "agent should know when to reconsider" principle, and the learned update_context tool is a notable departure from externally-imposed context management. For researchers tracking agentic AI, the key takeaway is that verification as a transition signal — not just a terminal filter — substantially changes agent control flow and enables systematic multi-constraint reasoning. This work also contributes a practical template for training long-horizon agents: synthetic verifiable task construction, key-step focused credit assignment, and progressive capability staging are all transferable patterns.