ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

RL with verifiable rewards for complex video reasoning in VLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ReWatch-R1 addresses the data bottleneck limiting RLVR-based video reasoning by introducing ReWatch, a large-scale dataset built with a multi-agent ReAct pipeline that synthesizes temporally grounded captions, hard multi-hop QA pairs, and video-anchored Chain-of-Thought traces. Post-training Qwen2.5-VL-7B with SFT on ReWatch followed by GRPO with a novel Observation & Reasoning (O&R) reward achieves state-of-the-art on five complex video reasoning benchmarks.

Problem

Existing SFT+RLVR pipelines for video reasoning are bottlenecked by low-quality training data: prior datasets feature holistic non-timestamped captions, simple perception-based QA answerable from short clips or text priors, and visually unfaithful CoT that relies on commonsense rather than grounded video evidence. This prevents models from learning true temporal, multi-hop reasoning and leaves RL without a reliable process-level reward signal, causing hallucination and poor logical fidelity.

Method

Data construction (ReWatch, 3 stages): 1. Hierarchical Video Captioning (ReWatch-Caption-10k): Videos are semantically segmented by an LVLM at low frame rate; each segment is captioned at high frame rate with per-event relative timestamps \(\tau_{ij}\), which are realigned to absolute times \(t_{ij} = t_i^{\text{start}} + \tau_{ij}\). 2. High-Difficulty QA Generation (ReWatch-QA-170k): A contrastive prompting strategy generates QA pairs answerable from the detailed caption \(C_{\text{detail}}\) but not from a concise summary \(C_{\text{sum}}\). Three-layer filtering enforces answer correctness, text-bias elimination (LLM ensemble below threshold \(\theta_{\text{text}}\)), and summary-bias elimination (below \(\theta_{\text{sum}}\)), yielding 85k pairs rewritten into 170k multiple-choice items. 3. Multi-Agent CoT Synthesis (ReWatch-CoT-135k): A ReAct loop between a Reasoner Agent \(A_R\) (generates thoughts \(T_t\) and actions \(\text{Act}_t\)) and an Observer Agent \(A_O\) (queries the textual caption \(C_{\text{detail}}\) via segment_retrieval and segment_query actions). The structured trajectory \(\{(T_t, \text{Act}_t, \text{Obs}_t)\}\) is converted to a natural-language CoT with explicit <action> and <observation> tags.

Post-training (ReWatch-R1): - SFT: Multi-task on all three ReWatch components with composite loss \(\mathcal{L}_{\text{SFT}} = \mathcal{L}_{\text{Cap}} + \mathcal{L}_{\text{QA}} + \mathcal{L}_{\text{CoT}}\). - RL (GRPO): Novel Observation & Reasoning (O&R) reward: $\(r_{O\&R} = r_{\text{acc}} \times (1 + r_{\text{obs}} + r_{\text{rea}}) + r_{\text{fmt}}\)$ where \(r_{\text{obs}}\) scores intermediate observations against \(C_{\text{detail}}\), and \(r_{\text{rea}}\) checks whether the extracted action-observation pairs alone suffice to answer the question correctly — penalizing hallucination at the process level, not just the outcome.

Key Contributions

• Multi-stage agentic pipeline producing ReWatch (10k captions, 170k QA pairs, 135k CoTs) with temporal grounding, contrastive difficulty, and video-anchored reasoning traces.
• O&R reward for RLVR that evaluates both final-answer correctness and factual fidelity of intermediate observations and reasoning steps.
• ReWatch-R1 (7B), achieving SOTA on five video reasoning benchmarks at both 192- and 384-frame settings.

Results

• Average across 5 benchmarks (192 frames): ReWatch-R1 + O&R: 35.51% vs. best comparable baseline GRPO-CARE: 30.14%, VersaVid-R1: 30.88%, Qwen2.5-VL-32B (no thinking): 35.71%.
• ReWatch-R1-SFT alone (33.25%) outperforms Video-R1-SFT (29.74%) and LongVideoReason-SFT (26.31%) using the same training configuration.
• RL gain: SFT→RL improves 33.25%→35.51% (192f) and 32.80%→35.78% (384f).
• Ablation — CoT data quality: Replacing ReWatch-CoT with Video-R1-CoT drops overall accuracy from 43.3% to 38.9% (combined reasoning+understanding suite).
• Ablation — QA data quality: RL on baseline QA (Video-R1-QA + LongVideoReason-QA): 34.3% reasoning; RL on ReWatch-QA: 34.9% reasoning, with larger gains on understanding tasks.
• Dataset complexity: ReWatch-QA requires 3.31 avg reasoning steps vs. 1.82 for Video-R1-QA; responses are 398.75 tokens vs. 205.74; Video-R1-QA shows high text-only solvability (~68.9%) vs. ReWatch-QA (~29.4%).

Limitations

• The Observer Agent queries textual captions rather than raw video frames during synthesis; scalability and fidelity depend on caption quality as a visual proxy.
• Evaluated on a single backbone family (Qwen2.5-VL-7B); generalization to other LVLM architectures is not demonstrated.
• The O&R reward relies on an LLM judge (\(M_{\text{judge}}\)) and an inference model (\(M_{\text{infer}}\)) at RL time, adding computational overhead and introducing judge model dependency.
• Performance on video understanding benchmarks (vs. reasoning) is reported only in the appendix; gains there are less emphasized.

Relevance to Agentic AI / LLM Agents

The Multi-Agent ReAct framework at the core of ReWatch directly instantiates a two-agent system (Reasoner + Observer) with a structured Thought-Action-Observation loop — a canonical agentic pattern — applied to synthetic data generation rather than task execution, which is a novel use of agent architectures. The O&R reward formalizes process-level grounding as a training signal, connecting to ongoing work on rewarding intermediate reasoning steps in agent traces (process reward models, verifiable chain-of-thought). For researchers tracking LLM agents, this paper demonstrates that agentic data synthesis pipelines can bootstrap higher-quality training corpora than static curation, and that rewarding the quality of observations and actions — not just outcomes — is critical for grounding agents in perceptual evidence. The approach is directly transferable to other modalities where agents must interleave retrieval and reasoning over structured external context.