Skip to content

Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

REVERSE (REtrospective VERification and SElf-correction) is a unified framework that adds hallucination-aware training and on-the-fly self-correction to VLMs without requiring external verifier models. It introduces three special tokens (<SPAN>, </CN>, </UN>) to mark phrase-level confidence, and a retrospective resampling mechanism that backtracks and rewrites uncertain spans during decoding. On LLaVA-v1.5-7B it reduces CHAIRi by 12% over the best prior method and improves HaloQuest accuracy by 34%.

Problem

Existing hallucination mitigation methods fall into two siloed paradigms: generation-adjustment methods (training-free logit editing or training-based fine-tuning) have no correction mechanism once tokens are emitted, while post-hoc verification pipelines require multiple external models (e.g., GPT-4 as verifier) and typically refuse outputs rather than refining them. Neither paradigm unifies detection and iterative correction within a single model.

Method

Hallucination-aware training. A 1.3M semi-synthetic instruction-tuning dataset is constructed by augmenting LLaVA-v1.5-665k with rule-based and GPT-4o-mini-generated negative phrases. Every groundable phrase is wrapped with <SPAN>…</CN> (confident) or <SPAN>…</UN> (hallucinated); tokens inside <SPAN>…</UN> spans are masked in the cross-entropy loss (indicator 1_Hall(i)=0) so the model learns confidence without reinforcing hallucinated content.

Retrospective resampling at inference. After each span closure, the model computes P(</UN>) over preceding tokens. If it exceeds threshold τ, the model: 1. Backtracks to the most recent </CN> checkpoint and applies rejection sampling (resampling at incrementally increased temperature until P(</UN>) < τ or K=10 local attempts are exhausted). 2. Falls back to the last sentence boundary if local retries fail. 3. Applies query rewriting, injecting a hint (Hint: potential incorrect phrases → <placeholder>) into the prompt to guide regeneration from that backtrack point. 4. Terminates after N=50 total correction attempts and flags the unresolved hallucination.

Special tokens are stripped from the final output; only verified text is returned to the user.

Key Contributions

  • REVERSE: the first single-model framework unifying generation, verification, and iterative correction for VLM hallucination.
  • Three vocabulary tokens (<SPAN>, </CN>, </UN>) enabling explicit, phrase-level confidence tagging as an inline classifier.
  • A modified cross-entropy loss that masks hallucinated span targets to avoid reinforcing language priors on ungrounded content.
  • A 1.3M-sample hallucination-aware instruction-tuning dataset (6.8M QA turns; publicly released) with paired positive/negative phrases and query-rewriting hints.
  • An adjustable threshold Ï„ giving users a single, interpretable knob to trade off hallucination reduction against content coverage.

Results

  • CHAIR-MSCOCO (LLaVA-v1.5-7B): CHAIRi 6.1 vs. 11.0 (HA-DPO, best prior); CHAIRs 13.6 vs. 38.2 at Ï„=0.0003. At Ï„=0.003: CHAIRi 10.3, CHAIRs 37.0, outperforming all baselines.
  • AMBER-G (LLaVA-v1.5-7B, Ï„=0.003): CHAIR 6.0, Hall 30.4, Cog 3.0 — best across compared methods.
  • HaloQuest: REVERSE(Ï„=0.003) achieves 30.7% avg. accuracy on LLaVA-v1.5 vs. 23.9% for HALVA (+6.8 pp); Ï„=0.0003 reaches 32.3%. On LLaVA-MORE: 36.7% vs. 22.8% (DoLA). On Qwen2.5-VL: 45.1% vs. 33.5% (base).
  • MMHal-Bench (LLaVA-v1.5-7B): Score 2.56 (Ï„=0.003) / 3.28 (Ï„=0.0003) vs. 2.25 (HALVA); hallucination rate 0.47/0.30 vs. 0.54.
  • Efficiency: 37% of samples require zero backtracking; >50% of corrected samples converge in one round; full N=50 rounds costs ~3.05× tokens vs. baseline; inline verification adds negligible overhead (no auxiliary model).
  • Generalizes across LLaVA-v1.5-7B, LLaVA-MORE-8B, and Qwen2.5-VL-3B backbones.

Limitations

  • Does not improve performance on discriminative (Yes/No) VQA tasks; backtracking provides limited benefit where outputs are single tokens.
  • Increased conservatism at low Ï„ reduces accuracy on visually challenging questions by refusing speculative but potentially correct answers.
  • Dataset uses GPT-4o-mini for augmentation, introducing potential bias and coverage gaps; inherits MS-COCO biases (gender, geography, race).
  • Dataset lacks edge-case coverage (false premises, insufficient context) present in newer benchmarks.
  • Inference cost scales with correction rounds (~3× token overhead at N=50); KV-cache reuse is noted but not fully evaluated.
  • Confidence threshold Ï„ is not auto-calibrated and must be set per model backbone due to VLM score miscalibration.

Relevance to Vision-Language Models

REVERSE directly addresses one of the most persistent failure modes in deployed VLMs—object hallucination—by converting the model itself into its own verifier through vocabulary augmentation and targeted loss masking, eliminating the common dependency on external oracle models. The explicit confidence-token paradigm introduces a novel training signal that resembles DPO in contrasting positive/negative phrase pairs, opening a direction for self-supervised grounding. The controllable τ parameter is particularly relevant to practitioners deploying VLMs in safety-critical settings (autonomous driving, assistive technologies), as it provides a calibrated, interpretable handle on the precision–recall trade-off. This work also contributes a large-scale public dataset for hallucination-aware fine-tuning, which is a resource gap the VLM community has actively sought to fill.