ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; RLVR agentic data synthesis for complex video reasoning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ReWatch-R1 addresses the data bottleneck limiting RLVR-based video reasoning in LVLMs by introducing a three-stage agentic pipeline that synthesizes temporally dense captions, hard multi-hop QA pairs, and video-grounded Chain-of-Thought data. A novel Observation & Reasoning (O&R) reward augments GRPO by jointly scoring final answer correctness and the factual grounding of intermediate reasoning steps against video content. The resulting model, built on Qwen2.5-VL-7B, achieves state-of-the-art average accuracy across five challenging video reasoning benchmarks at both 192-frame and 384-frame settings.

Problem

Existing SFT+RLVR pipelines for video reasoning fail because their training data contains three compounding flaws: (1) holistic, untimestamped captions that destroy temporal structure; (2) simple, perception-based QA answerable from short clips or text priors alone; and (3) visually unfaithful CoT that relies on commonsense reasoning rather than video-grounded evidence. Without high-quality, temporally-anchored CoT data and a reward signal that evaluates reasoning process quality, RL cannot penalize hallucination or teach true multi-hop temporal reasoning.

Method

Data construction (ReWatch dataset):

1. ReWatch-Caption-10k — Hierarchical Dynamic Frame-Rate pipeline: long videos are semantically segmented by a low-frame-rate LVLM into ~10-min coherent segments \(S = \{s_1, \ldots, s_k\}\); each segment \(s_i\) is recaptioned at high frame rate producing timestamped event descriptions \(D^{rel}_i = \{(c^i_j, \tau^i_j)\}\); relative timestamps are realigned to absolute ones via \(t^i_j = t^{start}_i + \tau^i_j\).

2. ReWatch-QA-170k — Contrastive prompting: a QA generator \(M_{qa}\) is given both the detailed caption \(C_{detail}\) and a lightweight summary \(C_{sum}\), producing questions answerable from \(C_{detail}\) but not from \(C_{sum}\). A three-tier filter eliminates factually wrong answers (F1), text-bias answerable from general knowledge (F2), and summary-bias answerable from \(C_{sum}\) (F3), retaining 85k pairs that are then rewritten into 170k multiple-choice questions.

3. ReWatch-CoT-135k — Multi-Agent ReAct framework: a Reasoner agent \(A_R\) emits thoughts and actions (Eq. 9); an Observer agent \(A_O\) executes two caption-lookup actions — segment_retrieval(query) and segment_query(timestamp) — against \(C_{detail}\) (Eq. 10). The resulting structured trajectory is converted to a natural-language CoT string with explicit <action> and <observation> tags.

Post-training: - SFT on all three datasets with composite loss \(\mathcal{L}_{SFT} = L_{Cap} + L_{QA} + L_{CoT}\), jointly training captioning, direct answering, and step-by-step reasoning modes. - RLVR via GRPO with the O&R reward: $\(r_{O\&R} = r_{acc} \times (1 + r_{obs} + r_{rea}) + r_{fmt}\)$ where \(r_{obs}\) averages per-step factual consistency of observations against \(C_{detail}\), and \(r_{rea}\) is the accuracy of answering \(Q\) solely from the parsed actions and observations.

Key Contributions

• ReWatch dataset: 10k captions, 170k QA pairs, and 135k CoT traces synthesized via a multi-stage agentic pipeline with explicit temporal grounding.
• Multi-Agent ReAct CoT synthesis: a Reasoner+Observer two-agent loop that externalizes the "re-watching" process, producing verifiable thought-action-observation trajectories from text-based video caption proxies.
• O&R reward: decomposes RLVR reward into accuracy, observation fidelity, and reasoning sufficiency, directly penalizing hallucinated intermediate steps.
• ReWatch-R1: Qwen2.5-VL-7B post-trained with SFT+RLVR, achieving SOTA on VCR-Bench, MINERVA, Video Holmes, VideoMathQA, and CG-AV-Counting.

Results

• 192-frame average (5 reasoning benchmarks): ReWatch-R1 + O&R: 35.51% vs. best comparable prior (LongVideoReason-RL†: 31.58%, Video-R1-RL†: 30.28%, VideoRFT: 30.83%, GRPO-CARE: 30.14%); outperforms Qwen2.5-VL-32B (35.71%) near-parity at 7B scale.
• 384-frame average: ReWatch-R1 + O&R: 35.78% vs. Qwen2.5-VL-32B: 36.39%.
• SFT-only baseline: ReWatch-R1-SFT (33.25%) already surpasses Video-R1-SFT (29.74%) and LongVideoReason-SFT (26.31%) at 192 frames.
• Ablation — CoT data quality: replacing ReWatch-CoT with Video-R1-CoT drops overall accuracy from 35.5% to 34.9%; removing SFT entirely collapses to 26.4%.
• Ablation — QA data quality: RL on baseline QA (34.3% reasoning) vs. RL on ReWatch-QA (34.9% reasoning), with overall improvement from 42.0% to 43.3%.
• Dataset complexity: ReWatch-QA requires ~3.31 reasoning steps vs. 1.82 for Video-R1-QA; average response length 398.75 vs. 205.74 tokens; Video-R1-QA shows 68.9% text-only answerable rate vs. 29.4% for ReWatch-QA.

Limitations

• Observer agent \(A_O\) retrieves observations from textual captions \(C_{detail}\) rather than raw video frames; the pipeline's visual fidelity is bounded by captioning quality and cannot capture visual details not encoded in text.
• CoT synthesis scalability depends on the quality and coverage of the upstream hierarchical captioner; errors in Stage 1 propagate through Stages 2 and 3.
• O&R reward computation requires running additional LLM judges (\(M_{judge}\), \(M_{infer}\)) per rollout, increasing RL training cost.
• Evaluation is limited to multiple-choice benchmarks; open-ended or grounded video QA performance is not assessed.
• Models are all ~7–9B parameters; scaling behavior of the approach to larger models is not studied.

Relevance to Vision-Language Models

ReWatch-R1 directly addresses a fundamental bottleneck for video LVLMs: the absence of temporally grounded, multi-hop CoT training data necessary for RLVR to succeed beyond image domains. The O&R reward mechanism offers a principled extension of process-reward modeling to video, penalizing hallucination at the intermediate reasoning step rather than only at the final answer — a paradigm applicable broadly to multimodal chain-of-thought training. The multi-agent ReAct data synthesis framework is a scalable, modality-agnostic recipe that could extend to other grounded-reasoning domains (e.g., document QA, 3D scene understanding). For VLM researchers, this work highlights that data architecture — not just model scale or RL algorithm — is the primary lever for complex temporal reasoning.