Skip to content

VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VideoHallu is a 3,233-QA benchmark of synthetic videos (generated by Veo2, Sora, Kling, etc.) deliberately violating physics and commonsense, designed as negative-control tests for VLMs. State-of-the-art models including GPT-4o and Gemini-2.5-Pro score only ~50% — barely above random — revealing that they rely on language priors rather than actual visual grounding. GRPO fine-tuning on mixed real+synthetic data improves VideoHallu accuracy without degrading real-world benchmark performance.

Problem

Existing VLM evaluations use positive-control tests on real-world videos that closely match training distributions, making it impossible to distinguish genuine visual understanding from memorized visual-language correlations. Negative-control tests using physically impossible or logically inconsistent content are rare in the real world and largely absent from prior benchmarks. This gap leaves it unclear whether VLMs reason visually or simply pattern-match to linguistic priors.

Method

The authors construct VideoHallu via a two-stage pipeline. First, five human experts write 141 adversarial prompts specifically designed to elicit physics/commonsense violations; these prompts are fed to seven video generators (Sora, Veo2, Runway Gen2, Kling, Pixverse, Lavie, CogVideo) to produce 987 synthetic videos. Second, experts manually review each video and craft adversarial QA pairs categorized into four groups: (a) Physics (gravity, momentum, state transitions), (b) Commonsense Reasoning (general knowledge + chain-of-thought), (c) Spatial-Temporal Consistency (abrupt entity changes across frames), and (d) Text-Visual Alignment (entity identity vs. prompt). For post-training, they compare SFT (negative log-likelihood) against GRPO (Group Relative Policy Optimization) using ROUGE-{1,2,L} averaged as the reward signal, on a mixture of 800 VideoHallu training pairs and 2,000 real-world Video-R1 samples, training Qwen2.5-VL-7B and LLaVA-OneVision on 8×A100 80G GPUs for one epoch.

Key Contributions

  • VideoHallu dataset: 3,233 expert-annotated QA pairs (800 train / 908 val / 1,525 test) over synthetic videos with controlled physics and commonsense violations across four categories.
  • Benchmark of 17 VLMs: Systematic evaluation showing even the strongest models (Gemini-2.5-Pro, GPT-4o, Qwen2.5-VL-72B) fail near chance level (~50%) on counterintuitive synthetic video understanding.
  • Finding that RL/CoT trained models (Video-R1, VideoChat-R1) do not improve over their base model on VideoHallu, and can even worsen commonsense/alignment, because chain-of-thought amplifies reliance on language priors.
  • GRPO > SFT for OOD synthetic reasoning: GRPO on mixed real+synthetic data yields larger accuracy gains than SFT on both Qwen2.5-VL-7B and LLaVA-OneVision.
  • Synthetic data is necessary: Real-data-only GRPO gives only +0.07% over baseline; combined data yields the best result.

Results

  • Overall accuracy on VideoHallu test set: All models cluster near 50%; best open-source model is Video-R1 at 53.64%; Gemini-2.5-Pro at 53.05%, GPT-4o-mini at 50.36%.
  • R1-fine-tuned models vs. base: Video-R1 (53.64%) only marginally improves over Qwen2.5-VL-7B (52.98%); VideoChat-R1 (48.85%) underperforms the base model — CoT training hurts on synthetic OOD content.
  • GRPO vs. SFT (Qwen2.5-VL-7B): GRPO reaches 57.69% overall vs. SFT 51.02%; similarly for LLaVA-OneVision: GRPO 41.38% vs. SFT 38.82%.
  • Data ablation (GRPO on Qwen2.5-VL-7B): Real-only 53.05%, Synthetic-only 55.41%, Combined 57.69%.
  • Real-world benchmarks preserved: Combined GRPO training reaches 61.3 on MMVU and 70.1 on MVBench vs. base 58.7 / 69.6 — no degradation.

Limitations

  • Domain gap between generated synthetic videos and real-world visual complexity may reduce transfer of improvements.
  • Limited coverage of rare physical edge cases and long-horizon causal chains.
  • High annotation cost: expert labeling, adversarial prompt crafting, and GRPO fine-tuning are expensive and hard to scale.
  • QA-based scoring (LLM-as-a-Judge with ROUGE rewards) may not capture visual grounding quality or uncertainty calibration.
  • Training set is small (800 synthetic samples), necessitating mixture with real data; pure synthetic training is data-hungry.

Relevance to Vision-Language Models

VideoHallu directly probes a critical weakness in VLMs: the tendency to override visual evidence with memorized linguistic priors, which is an underexplored failure mode distinct from the entity/event hallucinations studied in prior work like VideoHallucer. The finding that RL/CoT post-training (Video-R1, VideoChat-R1) fails to help — and can harm — on OOD synthetic content challenges the prevailing narrative that R1-style training universally improves VLM reasoning. The GRPO-with-synthetic-data result suggests a targeted mitigation path: curated negative-control data is necessary for VLMs to develop visually grounded rather than language-prior-driven reasoning, with implications for robustness in downstream applications like autonomous driving simulation and synthetic content moderation.