Skip to content

WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; VL deep research agent navigates visual web for complex information-seeking

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

WebWatcher is a multimodal deep research agent that jointly reasons over visual and textual information using a suite of web tools, trained via synthetic trajectory generation followed by SFT cold-start and GRPO reinforcement learning. The authors also introduce BrowseComp-VL, a challenging entity-obfuscated VQA benchmark requiring multi-hop cross-modal retrieval. WebWatcher-32B achieves state-of-the-art on LiveVQA (58.7%) and MMSearch (55.3%) while outperforming all open-source agents on BrowseComp-VL.

Problem

Existing deep research agents are text-centric: VL agents excel at perception-level visual tools (OCR, bounding box) but lack deep textual reasoning, while search-only agents fail on tasks requiring visual grounding or interactive browsing. No open-source agent can jointly handle complex multi-hop visual+textual retrieval in a flexible, non-template-driven way. Additionally, existing VQA benchmarks are shallow (≤2-hop) and perception-focused, providing insufficient evaluation difficulty for research-grade multimodal agents.

Method

Data construction: Multi-hop VQA pairs are built by random-walking hyperlink graphs from arXiv/GitHub/Wikipedia to form Level 1 (explicit entity) and Level 2 (entity-obfuscated) QA pairs. Text-to-VQA conversion replaces the target entity with a visual reference token and retrieves web images via Google SerpApi. A two-stage selector–examiner pipeline (GPT-4o) filters on semantic alignment and visual answerability.

Trajectory generation: Given a VQA instance \((I, q, a)\), GPT-4o constructs ReAct-style tool-use trajectories \(\tau = \{(t_0, o_0), \ldots, (t_L, o_L)\}\) from five tools—Web Image Search, Web Text Search, Visit (Jina), Code Interpreter, and OCR. Trajectories are filtered by answer correctness, step-wise logical consistency (GPT-4o judge), and a minimum of three tool calls.

Post-training: 1. SFT cold start — maximizes \(\sum_{i,l} \log P_\theta(t_l^{(i)} \mid I^{(i)}, q^{(i)}, t_{<l}^{(i)}, o_{<l}^{(i)})\) on 8,000 filtered trajectories. 2. GRPO RL — generates groups of \(K=16\) rollouts, computes group-relative advantage \(A_{\text{rel}}(\tau^{(i)}) = R^{(i)} - \frac{1}{K}\sum_j R^{(j)}\), and optimizes a clipped surrogate with KL penalty \(\beta D_{\text{KL}}(\pi_\theta \| \pi_{\theta_\text{old}})\). Reward \(R = 0.2 r_f + 0.8 r_a\) combines binary format compliance \(r_f\) and LLM-judged semantic accuracy \(r_a\).

Base models: Qwen2.5-VL-7B and Qwen2.5-VL-32B.

Key Contributions

  • WebWatcher: an open-source multimodal deep research agent with flexible multi-tool reasoning (image search, text search, browsing, code, OCR), trained end-to-end via SFT+GRPO.
  • BrowseComp-VL: a 399-sample (199 L1 + 200 L2) benchmark extending BrowseComp to visual modality with entity-obfuscated, multi-hop cross-modal queries verified by PhD-level annotators.
  • Synthetic VQA pipeline: scalable method to convert existing text QA datasets into high-quality multimodal examples with authentic web images and two difficulty levels.
  • Trajectory generation pipeline: automated ReAct-style tool-use trajectory synthesis with three-stage quality filtering (correctness, consistency, tool-use depth).

Results

  • HLE (VL subset): WebWatcher-32B avg 13.6%, vs. best prompt workflow Gemini-2.5-flash 10.6% and OmniSearch 9.3%; achieves 33.8% in Biology (top among all systems).
  • BrowseComp-VL: WebWatcher-32B avg 27.0% (L1: 28.4%, L2: 25.0%) vs. OmniSearch 16.3%, best prompt workflow (o3) 24.9%. Most baselines score below 20%.
  • LiveVQA: WebWatcher-32B 58.7%, state-of-the-art.
  • MMSearch: WebWatcher-32B 55.3%, state-of-the-art.
  • SimpleVQA: WebWatcher-32B 59.0%, competitive with prompt workflows.
  • Human vs. agent on BrowseComp-VL: Humans achieve L1 33.2% / L2 18.0%; WebWatcher-32B achieves L1 28.4% / L2 25.0%, surpassing humans on L2.
  • Ablation: trajectories with ≥3 tool calls yield best Pass@3 (19.09%), and performance plateaus beyond 3 tool calls.

Limitations

  • WebWatcher-32B still trails reasoning-scale proprietary models (o4-mini, Gemini-2.5-Pro) on HLE overall despite being parameter-efficient.
  • Training data construction relies heavily on GPT-4o for trajectory annotation and quality filtering, incurring proprietary API cost and potential bias.
  • BrowseComp-VL is limited to 399 samples; benchmark coverage of visual domains is weighted toward its construction sources (Wikipedia, web images).
  • Tool set is fixed at inference; the agent cannot dynamically expand to new tool types without retraining.
  • Ablation study uses only HLE; tool-call count sensitivity across other benchmarks is not analyzed.

Relevance to Vision-Language Models

WebWatcher directly extends the frontier of VLMs from static image-question answering to agentic, multi-step cross-modal retrieval, demonstrating that post-training small open-source VLMs (7B/32B) with tool-augmented trajectories and GRPO can match or exceed much larger proprietary systems on knowledge-intensive tasks. The BrowseComp-VL benchmark fills a critical gap: existing VQA benchmarks test perception, but this tests whether VLMs can plan, browse, and synthesize evidence across modalities—a capability increasingly demanded as VLMs are deployed in agentic pipelines. The trajectory generation and SFT+GRPO recipe is directly applicable to other VLM backbones and is relevant to anyone studying tool-use learning, multimodal reasoning, or web agents built on VLM foundations.