Skip to content

HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models

🕒 Published (v1): 2026-06-25 15:50 UTC · Source: Arxiv · link

Why this paper was selected

Systematic harmful-video benchmark for LMMs; fills critical safety evaluation gap for video VLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

HarmVideoBench is a 1,379-video, 4,137-question diagnostic benchmark that decomposes harmful video understanding into three hierarchical reasoning levels rather than binary classification. Evaluation of 19 state-of-the-art large multimodal models reveals a structural performance cliff from surface cue detection to context-dependent reasoning. The authors also propose BCR, a lightweight retrieval-augmented inference method that raises macro accuracy from 61.7% to 84.4% on an 8B base model.

Problem

Existing harmful-video benchmarks reduce safety evaluation to binary (harmful/not-harmful) classification, ignoring the hierarchical nature of harm where implicit or culturally grounded content is entirely missed. Additionally, no existing benchmark requires or evaluates explanatory rationales, making current evaluations opaque black boxes where models can exploit superficial shortcuts without demonstrating genuine understanding.

Method

Benchmark construction: 1,379 videos sourced from HateMM, MultiHateClip, YouTube, and Bilibili are annotated with three multiple-choice questions each — one per reasoning tier. Qwen-2.5-VL-72B generates candidate QA items; a human-in-the-loop pipeline with four trained annotators and one senior adjudicator then accepts, rewrites, or rejects each item (44% required substantive rewriting; 77.41% of 5,344 candidates retained).

Three evaluation dimensions: 1. Observable Evidence — identify directly observable harmful cues (visual, acoustic, on-screen text, symbolic) resolvable from the clip alone. 2. Clip-Internal Meaning — infer the harmful proposition (mockery, glorification, incitement) by integrating multimodal clip evidence without external knowledge. 3. Beyond-Clip Reasoning — resolve questions requiring external cultural, historical, or socio-political context not present in the clip.

BCR (Boundary-Constrained Reasoning): Built on Qwen3-VL-8B. Three components: (1) reasoning-scope prediction — classifies each question as clip-grounded or beyond-clip; (2) selective context augmentation — retrieves top-\(k\) context snippets from a training-split memory bank only for beyond-clip questions; (3) boundary-constrained decoding — generates clip-grounded intermediate chain-of-thought before the final answer. Models receive 12 sampled keyframes plus ASR transcript; inference uses greedy decoding (\(T=0\)).

Key Contributions

  • HarmVideoBench: a three-tier diagnostic benchmark (1,379 videos, 4,137 MC questions) with human-in-the-loop curation covering violence, hate, self-harm, privacy, misinformation, and bullying across English and Chinese.
  • Systematic evaluation of 19 models (closed-source, open-source thinking, open-source non-thinking) revealing a monotonic reasoning-depth gap universal across architectures and scales.
  • BCR: a training-split-aligned, retrieval-augmented inference method with ablated scope control, selective retrieval, and boundary-constrained decoding.
  • Characterization of five recurring error categories (Evidence Miss, Meaning Collapse, Context Omission, Unsupported Harm Leap, Generic Moralization).

Results

  • Human ceiling: Observable Evidence 98.9% / Clip-Internal Meaning 97.5% / Beyond-Clip Reasoning 96.2% / Macro 97.5%.
  • Best closed-source (Gemini-3.1-Pro): 96.0% / 84.6% / 65.8% / 82.1% — 30.4-point gap to human on Beyond-Clip.
  • Best open-source thinking (Qwen3-VL-235B-A22B-Thinking): 95.1% / 82.6% / 62.5% / 80.1%.
  • Best open-source non-thinking (Qwen3-VL-235B-A22B-Instruct): 94.8% / 82.3% / 62.0% / 79.7%.
  • Smallest open-source (Qwen3-VL-8B-Instruct): 73.3% / 63.7% / 48.1% / 61.7%.
  • HarmVideo-SFT (8B fine-tuned): 93.5% / 83.0% / 64.8% / 80.5%.
  • BCR (8B, ours): 96.7% / 86.4% / 70.1% / 84.4% — best published macro average.
  • BCR ablations (Beyond-Clip): removing scope control → 66.0%, removing retrieval → 66.8%, removing boundary-constrained decoding → 66.5% (vs. 70.1% full BCR).
  • Performance degrades monotonically across all three tiers for every evaluated model, holding across architectures, scales, and harm categories.

Limitations

  • BCR is evaluated in a task-adapted (train-split memory bank) setting, not true zero-shot; gains may not transfer to unseen harmful domains.
  • 84.4% macro still leaves a ~13-point gap to human performance (97.5%), particularly on Beyond-Clip Reasoning (70.1% vs. 96.2%).
  • Benchmark is predominantly English and Chinese; cross-lingual analysis is noted but the paper is truncated before full results are presented.
  • Videos are sampled at only 12 frames, which may miss temporally distributed harmful cues.
  • Human annotator familiarity with benchmark policy means the "human baseline" overstates naive human performance.
  • Binary harmful umbrella label is used during collection despite multi-tier evaluation — fine-grained harm taxonomy is not systematically validated.

Relevance to Vision-Language Models

HarmVideoBench directly benchmarks the multimodal reasoning capabilities of LVLMs under adversarial safety conditions, exposing a fundamental gap in current VLMs between perceptual grounding (Observable Evidence, near-saturated) and world-knowledge integration (Beyond-Clip Reasoning, \(\sim\)30 points below human). The hierarchical evaluation design is methodologically transferable to any VLM benchmark where surface recognition and deep contextual inference must be disentangled. The BCR result demonstrates that lightweight retrieval augmentation at inference time can substantially close the context-gap for compact (8B) video-language models, relevant to ongoing work on RAG for VLMs. For researchers tracking VLMs, this benchmark reframes content moderation as a probe of cultural and contextual reasoning depth rather than visual detection, identifying the precise capability frontier current models fail to cross.