Skip to content

Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty

🕒 Published (v1): 2026-06-26 18:15 UTC · Source: Arxiv · link

Why this paper was selected

Counterfactual visual grounding for clinical hallucination detection; important safety work

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

A plug-and-play hallucination detector for medical LVLMs that audits arbitrary responses via a medical-domain-adapted Qwen-VL grounding verifier. It introduces counterfactual entity perturbation (e.g., "left kidney" → "right kidney") and an IoU-reweighted uncertainty score contrasting factual vs. counterfactual grounding, achieving both binary decisions and interpretable visual localization.

Problem

LVLMs hallucinate in clinical scenarios—describing findings, anatomy, or devices not supported by the input image—which can mislead clinicians. Existing hallucination detection methods either need internal model state access (hidden state-based, incompatible with black-box APIs) or output binary labels without visual localization (external verifier-based). Neither provides entity-level visual evidence for clinical auditing.

Method

  1. Entity extraction: Parse visually verifiable clinical entities (anatomy, lesions) from the LVLM response using GPT-4.1-mini.
  2. Visual grounding verifier: Fine-tune Qwen3-VL-2B with LoRA on 33,266 medical grounding pairs (from IMIS-Bench segmentation masks) to produce bounding boxes and confidence scores.
  3. Counterfactual construction: For each entity \(e\), generate a spatially exclusive counterfactual \(\tilde{e}\) using radiological knowledge (e.g., contralateral side, different organ class) that should not co-localize with \(e\).
  4. Uncertainty score: Compute factual confidence \(s_e^+\), counterfactual confidence \(s_e^-\), and their box overlap \(\text{IoU}_e\). The hallucination score is: $\(U_e = 1 - (s_e^+ - \text{IoU}_e \cdot s_e^-)\)$ An entity is hallucinated if \(U_e > \tau\) (tunable, optimal around \(\tau = 0.6\)). Two confidence variants: Monte-Carlo Sampling (pairwise mIoU over \(N\) stochastic inferences) or Internal Logit Probability (averaged softmax of predicted box logits).

Key Contributions

  • Vision-aware detection: provides both hallucination labels and entity-level visual evidence via grounding, enabling clinical traceability.
  • Counterfactual-driven uncertainty: contrasts factual and counterfactual grounding to suppress spurious alignments (ablation: without the counterfactual branch, MCS AUC drops from 0.8811 to 0.3572, below random).
  • Model-agnostic: requires no access to the target LVLM's internal states, works across 4 commercial/open LVLMs as a plug-in.
  • Lightweight verifier: 2B parameters vs. FaithScore's 13B.

Results

  • Evaluated on 1,904 image-report pairs (958 non-hallucinated, 946 hallucinated) from 4 LVLMs: MedGemma-27B, GPT-5.1, Gemini-3-Flash, Grok-4-Fast; 1,129 CT + 775 MRI scans.
  • MRI modality: Ours achieves up to 96.81% HDR (Grok-4) and 93.88% precision—substantially ahead of all baselines.
  • CT modality: Ours maintains highest precision across backbones (up to 68.00%), while UniHD has higher HDR but the lowest precision (56.84–58.95%).
  • AUC: MCS confidence = 0.8811, ILP confidence = 0.8763.
  • Ablation: removing the counterfactual branch drops MCS AUC to 0.3572 (below random); sign-flip correction yields 0.6428. Removing IoU reweighting reduces AUC moderately (0.8595 for MCS).
  • Optimal threshold \(\tau \approx 0.6\) balances HDR and precision.
  • Qualitative cases show the method flags a hallucinated "normal right kidney" (\(U_e = 0.815\)) where the counterfactual "thoracic cavity" grounds with high confidence.

Limitations

  • Verifier dependence: performance is bounded by the grounding verifier's accuracy; poor-quality images or limited verifier capability can still produce spurious localizations.
  • Static priors: counterfactual construction relies on hand-crafted spatially exclusive rules; does not handle visually entangled concepts (e.g., overlapping anatomies) where factual and counterfactual are not spatially disjoint.
  • No spatiotemporal scope: limited to static 2D images; does not extend to video, 3D volumes, or sequential imaging relationships.
  • Entity extraction overhead: depends on GPT-4.1-mini API for entity parsing, adding an external dependency and potential failure point.

Relevance to Vision-Language Models

This work directly tackles the clinically critical problem of VLM hallucination detection—a prerequisite for deploying VLMs in safety-critical medical settings. Its counterfactual grounding strategy is a novel departure from both hidden-state and binary-verifier paradigms, and the model-agnostic design makes it applicable to any VLM, addressing the cross-model generalization gap common in hallucination detection. For VLM researchers, the IoU-reweighted uncertainty score and the finding that a purely factual grounding baseline can perform below random without counterfactual correction are notable architectural insights.