Skip to content

Is Your Video Language Model a Reliable Judge?

🕒 Published (v1): 2025-01-01 · Source: ICLR · Venue: ICLR 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper systematically investigates whether video language models (VLMs) can reliably evaluate other VLMs, finding that most current VLMs are poor judges due to content-comprehension limitations and positive-rating bias. Collective thought approaches that aggregate judgments from mixed pools of reliable and unreliable judges do not improve — and can degrade — evaluation reliability compared to using GPT-4o alone.

Problem

Human evaluation of VLMs is inconsistent and unscalable, motivating VLM-as-judge pipelines. However, the reliability of VLMs as evaluators is poorly understood: individual models may hallucinate, overestimate candidates, or carry training biases. Ensemble/collective-intelligence approaches are an intuitive remedy, but whether they actually help when the judge pool is heterogeneous (mixing strong and weak models) had not been studied.

Method

The study uses the CVRR-ES dataset (2,400 open-ended QA pairs from 217 videos across 11 visual dimensions) as the evaluation substrate. Five VLM candidates (Video-LLaVA, LLaMA-VID, GPT-4o mini, Video-ChatGPT, mPLUG-Owl-Video) generate responses. Five VLM judges (same minus mPLUG-Owl, plus InternVL2 and GPT-4o) score those responses on a 1–5 scale. A reference-guided LLM Agent-Debate (GPT-4o + GPT-3.5 in text-only mode with gold reference answers) serves as the reliability ground truth. Agreement between each VLM judge and Agent-Debate is measured via Weighted Cohen's Kappa (quadratic weights, k=5).

Three evaluation regimes are compared: 1. Individual VLM judge — each model judges independently. 2. Collective Thought judge — GPT-4o aggregates all VLM judges' reviews alongside the video-QA content. 3. Mixture of Judges — per-dimension reliability gate selects only judges with κ above a threshold θ (or top-k) before GPT-4o consolidates.

Additionally, Video-LLaVA is fine-tuned to test whether improved comprehension alone translates to better judging reliability.

Key Contributions

  • First systematic study of VLMs evaluating VLMs on complex video understanding tasks.
  • Demonstrates that weaker VLMs as judges of stronger models produce unreliable evaluations due to insufficient reasoning capacity.
  • Shows collective thought aggregation from mixed-reliability pools does not improve — and often degrades — evaluation reliability vs. a strong solo judge.
  • Finds that fine-tuning for improved comprehension is insufficient to make a weak VLM a reliable judge; critical evaluation ability requires more than content understanding.
  • Identifies GPT-4o as the only tested VLM that achieves substantial agreement with the Agent-Debate reference method.

Results

  • Video-LLaVA Weighted Cohen's Kappa with Agent-Debate: −1.83 to 2.81 across dimensions (near-zero or negative = unreliable).
  • LLaMA-VID: κ range −4.15 to 3.70 (similarly unreliable).
  • GPT-4o individual judge: κ up to 60.38 (Social Context) and 55.18 (Visual Context), the highest of all tested judges.
  • Collective Thought (all judges → GPT-4o aggregator): average κ 2.72–42.53, lower than GPT-4o alone in several dimensions.
  • Mixture of Judges (reliability-gated): κ 2.72–60.70, no substantial improvement over collective thought and still below or comparable to GPT-4o solo.
  • Video-LLaVA post-fine-tuning: rating distribution remains skewed toward high ratings; κ only slightly improved, far below GPT-4o.
  • Results replicated on the VideoChatGPT dataset, confirming generality.

Limitations

  • Experiments limited to two datasets (CVRR-ES, VideoChatGPT) and a small set of model families; generalizability to other VLMs is uncertain.
  • Agent-Debate with gold reference answers is used as ground truth, but this assumes reference-guided text-only LLM debate is itself error-free — this is an unvalidated assumption.
  • Weighted Cohen's Kappa may not capture qualitative nuances in judgments.
  • GPT-4o as the aggregator in collective/mixture setups conflates the judge pool effects with the aggregator's own biases.
  • Computational cost of GPT-4o-based methods limits practical scalability.
  • Only scoring-based (1–5) evaluation is studied; pairwise ranking is derived from scores rather than tested independently.

Relevance to Vision-Language Models

This paper directly interrogates a core infrastructure question for VLM development: how to evaluate VLMs at scale without human annotators. The finding that most open-source/weaker VLMs are unreliable judges (with near-zero or negative κ) has direct implications for any pipeline that uses LLM-as-judge on video tasks — practitioners cannot simply swap in a lighter model. The weak-to-strong evaluation failure connects to the broader alignment literature and highlights that VLM capability gaps propagate into evaluation reliability, complicating both benchmarking and RLAIF reward modeling for video. The negative result on collective intelligence also pushes the field toward reliability-aware judge selection rather than naive ensembling.