Skip to content

VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning

🕒 Published (v1): 2026-07-03 03:50 UTC · Source: Arxiv · link

Why this paper was selected

RL trains multi-tool agentic reasoning for video; multimodal + harness design jointly optimized

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VideoSearcher is a closed-loop agentic framework that trains a VLM to perform Video Deep Research (VDR) — answering questions about videos by combining temporal localization, spatial focusing, and open-web multimodal search. It introduces BiSPO, a dual-branch RL algorithm that separately optimizes answer accuracy and tool-invocation behavior. A new benchmark, VideoSearch-QA (278 samples), is also released to evaluate this capability.

Problem

Existing video QA works assume closed-evidence settings (all answers reside in the video), while real-world queries require agents to use videos as visual anchors and retrieve external evidence from the web. Prior VDR attempts (e.g., VideoDR) convert visual cues to text before retrieval, discarding visual information. No training framework existed to teach a VLM to coordinate temporal grounding, spatial inspection, and multimodal web search over dynamic videos within a single reasoning trajectory.

Method

VideoSearcher equips a VLM (initialized from Qwen3-VL-8B-Instruct) with five tools: choose_frames (coarse temporal selection), find_frame (key-frame localization), zoom_in (region inspection), web_search, and image_search. Training proceeds in two stages:

  1. Cold-start SFT on 3,811 synthesized multi-tool trajectories built via an automated pipeline: core visual entities are extracted from image-text QA datasets using two LLMs (Qwen3.5-397B + Kimi-K2.5), matched to retrieved videos, rewritten into VDR instances, and filtered for quality.

  2. RL with BiSPO on 3,285 hard video-QA instances. BiSPO extends GSPO with sequence-level importance sampling (ratio \(s_i(\theta) = \exp(\bar{\ell}_i(\theta))\)) and two decoupled branches:

  3. Accuracy branch: group-normalized advantage \(A^\text{acc}_i = (R^\text{acc}_i - \mu^\text{acc}) / (\sigma^\text{acc} + \epsilon)\) over all \(G\) rollouts, combining LLM-judge answer score, format compliance, and tool-dependency reward.
  4. Tool branch: advantage \(A^\text{tool}_i\) computed only over the subset \(Q\) of correct rollouts, preventing incorrect trajectories from contaminating tool-behavior gradients. A bell-shaped gated reward encourages sufficient but not excessive tool use.
  5. Final loss: \(\mathcal{L}_\text{BiSPO} = w_\text{acc}\,\mathcal{L}_\text{GSPO}(A^\text{acc}) + w_\text{tool}\,\mathcal{L}_\text{GSPO}(A^\text{tool})\) with \(w_\text{acc}=1.0\), \(w_\text{tool}=0.15\).

Key Contributions

  • VideoSearcher: first closed-loop VDR agent unifying temporal localization, spatial focusing, and multimodal (image + web) search for dynamic videos.
  • BiSPO: dual-branch RL algorithm decoupling answer-accuracy optimization from tool-invocation optimization, with a correctness gate preventing tool-behavior gradients from incorrect rollouts.
  • VideoSearch-QA (VSQA): first manually annotated benchmark (278 samples) for video-grounded multimodal search, covering landmarks, geography, culture, and 2026 real-time events.
  • Video-centric data pipeline: automated synthesis of multi-tool VDR trajectories from image-text QA datasets, with dual-LLM entity validation and strict quality filters yielding 3,811 SFT trajectories.

Results

  • VideoSearcher-8B vs. Qwen3-VL-8B-Instruct baseline (agentic setting): +25pp on VideoDR (53 vs. 28), +20.86pp on VSQA, +20.47pp on MMSearch, +15.74pp on HR-MMSearch, +20.74pp on FVQA-test, +15.71pp on InfoSeek.
  • VideoSearcher-4B: competitive with VideoSearcher-8B on most benchmarks; outperforms all open-source agentic baselines including MMSearch-R1, DeepEyesV2, and Webwatcher-7B.
  • vs. proprietary models (agentic): VideoSearcher-8B (53.0 on VideoDR) exceeds GPT-4o (48.0) and approaches GPT-5.2 (58.0) and Gemini-3-Pro (65.11 on VSQA vs. VideoSearcher-8B's 51.80).
  • Direct-answer baselines (no tools) score 8–36 on VideoDR; agentic VideoSearcher-8B reaches 53, demonstrating that tool use is the primary driver of gains.

Limitations

  • Vision tower is frozen during both SFT and RL; the framework relies on the backbone's existing visual representations rather than improving them end-to-end.
  • VSQA benchmark is small (278 samples) and manually annotated, limiting statistical power.
  • RL rollouts require live tool execution (real web search, frame extraction), making training expensive and non-reproducible across different web states.
  • Tool-behavior optimization is skipped when fewer than two correct rollouts exist per prompt, potentially leaving hard samples undertrained for tool use.
  • Evaluation on VideoDR uses text-centric metrics inherited from that benchmark; the paper notes this is a limitation of the existing benchmark rather than VideoSearcher.

Relevance to Harnesses / Meta-Harnesses

VideoSearcher is a concrete instantiation of a task-specific agentic harness: it defines a closed action/observation loop, a curated tool set, and a training pipeline that teaches a model to orchestrate those tools end-to-end. The BiSPO training regime is notable for harness researchers because it operationalizes a meta-level objective — rewarding not just task success but how the agent uses its harness (tool invocation quality, sequencing, redundancy avoidance). The data-synthesis pipeline, which bootstraps trajectory examples by having stronger models operate the harness on hard samples, is a practical template for building harness-specific training corpora. The correctness-gated tool advantage is a general design principle applicable to any harness where tool cost or quality matters independently of final output correctness.