Skip to content

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ViCrit frames visual hallucination detection as a verifiable RL proxy task: a single, synthetically injected hallucination is hidden in a 200-word human-written caption, and the model must pinpoint the corrupted noun phrase via exact string match. This gives RL the same deterministic, scalable reward structure that made math/code RL successful, but for fine-grained visual perception. Training Qwen2.5-VL with GRPO on this task yields consistent gains across 10+ VL benchmarks, including out-of-domain visual math and chart reasoning.

Problem

RL with verifiable rewards has transformed LLM reasoning (math, code), but no analogous task existed for visual perception in VLMs. Exhaustive captioning requires paragraph-length outputs that are impossible to grade unambiguously; simpler VQA benchmarks probe only fragments of a scene and allow shallow perception to suffice. The gap is the absence of a vision-centric task that is simultaneously perceptually demanding (forces full-scene understanding) and automatically verifiable (produces a concise, deterministic response).

Method

Hallucination injection: Starting from PixMo-Cap's 196-word average human-annotated captions (384K images → 875K pairs), GPT-4 selects one noun phrase and replaces it with a visually similar, semantically plausible alternative across eight error types: object substitution, color/attribute flip, material swap, spatial relation error, count error, shape error, scene-text error, and condition/action error. The result is a minimally modified caption C̃ differing from the original by exactly one 2-word span.

RL training: Given image I and corrupted caption C̃, the model (Qwen2.5-VL-7B or 72B) must output the hallucinated span õ. Reward is binary exact string match (r_answer = +1 if prediction equals õ, else 0), with a format compliance reward (r_format). Final reward: r = 0.9·r_answer + 0.1·r_format. Training uses GRPO with all model parameters updated.

ViCrit-Bench: A 607-sample diagnostic benchmark curated from PixMo-Cap, covering 4 image domains (natural, document, scene-text-heavy, abstract) × 8 hallucination types, with manual caption validation and cross-annotator verification. Evaluated via exact string match accuracy.

Key Contributions

  • ViCrit proxy task: Converts fine-grained visual hallucination detection into a verifiable RL objective with binary exact-match reward, enabling scalable perception-centric RL training.
  • Cross-domain transfer: Gains generalize from natural-image training to abstract images, visual math, and charts—domains barely represented in PixMo-Cap training data—indicating the model learns a transferable "how to look" strategy rather than object memorization.
  • ViCrit-Bench: A 607-sample, category-balanced hallucination detection benchmark with strong monotonic correlation (r = 0.96) with overall VLM performance across 8 benchmarks, positioning it as a reliable diagnostic proxy.

Results

  • ViCrit-RL-72B overall: average VLM accuracy 59.78% → 63.16% (+3.38 pp) across 8 heterogeneous benchmarks vs. Qwen2.5-VL-72B-Instruct baseline.
  • MathVision (72B): 35.2% → 40.1% (+4.9 pp).
  • VLMsAreBlind (72B): 61.3% → 65.8% (+4.5 pp).
  • Charxiv reasoning (72B): 45.5% → 49.4% (+3.9 pp).
  • Hallucination — CHAIRS (72B): 26.4 → 21.0 (↓5.4); CHAIRI: 4.8 → 3.9 (↓0.9); MMHal: 3.82 → 3.91—outperforming all SOTA open-source VLMs on all three hallucination metrics.
  • ViCrit-RL-7B: average +2.4 pp over Qwen2.5-VL-7B; MMHal 3.77 surpasses multiple 72B-class models.
  • ViCrit-Bench ceiling: best closed-source model (o3) achieves only 47.7%; best open-source baseline (Qwen2.5-VL-72B) 42.4%; ViCrit-RL-72B reaches 43.0%.
  • Benchmark correlation: ViCrit-Bench overall score vs. average VLM performance yields r = 0.96 across all evaluated open-source models.

Limitations

  • Spatial and scene-text (Text) hallucination categories show accuracy drops post-RL training, attributed to class imbalance in the PixMo-Cap training distribution.
  • ViCrit-Bench gains for the 72B model are marginal relative to the 7B model; authors hypothesize the training data is insufficiently hard for the stronger base model.
  • Training data is 59% natural images; abstract and document images are underrepresented, potentially capping gains on those domains without targeted data augmentation.
  • ViCrit-Bench spatial accuracy remains extremely low (17.28% for ViCrit-RL-72B), and no mechanism to address spatial perception is proposed.
  • Hallucination injection relies on GPT-4 for data generation, introducing dependency on a proprietary model and potential systematic biases in perturbation style.

Relevance to Vision-Language Models

ViCrit directly addresses the core bottleneck in applying RLHF/GRPO to VLMs: the lack of vision-centric tasks with clean, unambiguous reward signals. By reframing exhaustive visual understanding as a short-answer detection problem, it unlocks scalable RL for fine-grained perception—a regime previously inaccessible because captioning outputs are unverifiable. The demonstrated cross-domain transfer to visual math and abstract reasoning suggests that perception-grounding RL objectives improve low-level visual representations in ways that propagate through downstream reasoning chains, complementing existing work on multimodal chain-of-thought and outcome-reward RL for VQA. ViCrit-Bench's r = 0.96 correlation with average VLM performance also makes it a compact, practical diagnostic for tracking perception quality across model versions.