An Open and Reproducible Deep Research Agent for Long-Form Question Answering¶
🕒 Published (v1): 2025-12-15 07:37 UTC · Source: Arxiv · Venue: NEURIPS · link
Ask a follow-up
Open an assistant pre-loaded with this paper's context.
💬 Ask ChatGPT✦ Ask Claude
TL;DR¶
An open-source deep research agent for long-form question answering that won the text-to-text track of the MMU-RAG competition at NeurIPS 2025. It combines iterative web retrieval (search → rerank → summarize) with a reasoning agent fine-tuned via DPO on LLM-judged preference pairs. All components use open models and an open web corpus, enabling full reproducibility.
Problem¶
Commercial deep research systems (e.g., OpenAI Deep Research, Google Gemini Deep Research) are proprietary and irreproducible. Existing open reproductions focus on short-form QA where answers can be verified against gold references; long-form open-domain QA with multi-aspect quality (clarity, insightfulness, factuality) remains underexplored.
Method¶
The system consists of two components:
Search Tool: A pipeline of (1) ClueWeb22-backed open search API (top-K=100 docs), (2) Qwen3-Reranker-0.6B selecting the top-N most relevant docs, and (3) Qwen3-Next-80B-A3B-Thinking as a summarizer that generates citation-marked summaries conditioned on the query and prior reasoning steps.
Research Agent: Qwen3-Next-80B-A3B-Thinking fine-tuned with DPO+LoRA (α=16, rank=16, β=0.5). Training data is constructed by: collecting 1,000 questions (500 Researchy Questions, 200 Natural Questions, 300 competition validation), generating 20 sampled answers per question from the untuned model, scoring each with o3-mini across clarity/insightfulness/factuality (KPR-based), and forming preference pairs by taking highest/lowest scoring answers. Three datasets at score-gap thresholds θ∈{0.3, 0.5, 0.7} yield 983/828/341 pairs respectively. DPO optimizes over both the reasoning chain and final answer; search tool tokens are masked out during training.
At inference, the agent first seeds its context with an initial search result, then iteratively issues up to 5 additional search queries before producing a long-form cited answer.
Key Contributions¶
- Fully open and reproducible deep research system (open LLM + open search corpus) winning NeurIPS 2025 MMU-RAG competition text-to-text track (Best Static Evaluation, open-source category)
- Rerank+summarize search module that provides citation-grounded, query-conditioned document summaries to the generator
- Multi-aspect DPO preference tuning pipeline using LLM-as-judge (o3-mini) across clarity, insightfulness, and factuality dimensions
- Publicly released codebase enabling community reproduction
Results¶
Evaluation on 100 held-out Researchy Questions (LLM-as-judge via o3-mini):
| Metric | VANILLA | OURS (θ=0.3) | Δ |
|---|---|---|---|
| Clarity (0–10) | 6.71 | 8.18 | +1.47 |
| Insightfulness (0–10) | 6.52 | 7.50 | +0.98 |
| Factuality (KPR, %) | 43.4 | 44.3 | +0.9 |
| Search Count | 1.03 | 1.20 | +0.17 |
| Citation Error Rate | 0.06 | 0.09 | +0.03 |
- All tuned models (θ∈{0.3, 0.5, 0.7}) consistently outperform VANILLA with only marginal differences among them
- Factuality gain is smallest; attributed to noisy click-based training signals and partial dataset coverage
Limitations¶
- Factuality improvements are marginal (+0.9 KPR) and attributed to noisy user-click-based signals only available for the Researchy Questions subset
- Evaluation relies entirely on LLM-as-judge (o3-mini); no human evaluation reported
- Citation error rate slightly increases after tuning (0.06→0.09)
- Agent is capped at 5 search queries per question, limiting depth on complex multi-hop questions
- No comparison against other open deep research systems (e.g., WebThinker, SFR-DeepResearch) on the same benchmark
Relevance to Agentic AI / LLM Agents¶
This paper is a direct instantiation of the tool-augmented LLM agent paradigm applied to information-seeking: an agent that plans, calls external tools (search), reasons over retrieved context, and synthesizes structured long-form outputs—a core agentic loop. The DPO-based training on multi-aspect LLM-judged preferences is particularly relevant to efforts on aligning agentic behavior (not just final answers but intermediate reasoning quality). The rerank-then-summarize search tool design addresses a key challenge in agentic retrieval: reducing context noise before it reaches the reasoning model. Its open release fills an important reproducibility gap in the deep research agent space, enabling the community to study and extend iterative retrieval-reasoning systems.