Skip to content

Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification

🕒 Published (v1): 2026-06-17 02:14 UTC · Source: Arxiv · Venue: MICCAI 2026 · link

Why this paper was selected

MICCAI 2026; counter-evidence verification for hallucinations in high-stakes medical VLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CoEV (Counter-Evidence Verification) is a training-free, plug-and-play framework for detecting and correcting hallucinations in medical VLMs by performing counterfactual visual interventions — masking evidence regions and observing whether claims persist. It introduces a four-quadrant diagnostic taxonomy that jointly captures textual factuality and visual grounding, then uses this signal for post-hoc report refinement and VQA correction.

Problem

Existing hallucination detection methods rely on textual similarity metrics (BLEU, ROUGE) or uncertainty estimation, neither of which verifies whether model outputs are causally grounded in specific visual regions. Attention/saliency maps only show statistical correlations, not evidence dependence. Medical VLMs can produce fluently written but clinically incorrect reports, creating patient safety risks that coarse-grained metrics cannot reliably catch.

Method

CoEV operates in three stages:

  1. Diagnosis-Grounded Module (DGM): A semantic parser extracts positive factual claims \(C_{pos}\) from generated text; a grounding model \(G\) localizes bounding box \(B_i\) with confidence \(s_i\) for each claim \(c_i\), yielding grounding indicator \(\delta_i \in \{0,1\}\).

  2. Counter-Evidence Consistency Evaluation (CECE): For each grounded claim, a counter-evidence image \(I'_i = M(I, B_i)\) is created by blacking out \(B_i\). The VLM is re-queried on \(I'_i\); if claim \(c_i\) persists in the modified output (\(\phi(c_i, T'_i)=1\)), grounding attribution \(z_i=0\) (prior-driven); otherwise \(z_i=1\) (visually dependent).

  3. Diagnosis Mapper (DM): Factual correctness \(f_i\) and grounding attribution \(z_i\) are combined into a four-quadrant map \(D(f_i, z_i) \in \{Q1, Q2, Q3, Q4\}\): Q1 = correct+grounded; Q2 = incorrect+grounded (language bias); Q3 = incorrect+ungrounded (visual misinterpretation); Q4 = correct+ungrounded. Claims in Q2/Q3/Q4 are hallucinated (\(y_i=1\)).

For correction, flagged sentences in reports are rewritten via an LLM prompted to produce the opposite clinical meaning; VQA answers in Q2/Q3 are flipped. No retraining is required.

Key Contributions

  • Training-free, architecture-agnostic post-hoc hallucination detection and correction applicable to arbitrary medical VLMs.
  • Counterfactual visual masking to test causal dependence of claims on localized image regions, going beyond attention visualization.
  • Four-quadrant diagnostic taxonomy distinguishing four failure modes (language bias, visual misinterpretation, total failure, factually correct but ungrounded).
  • Demonstrated gains on both open-ended MRG and closed-ended Med-VQA tasks across four datasets and four VLM architectures (InternVL3-2B, Qwen2-3B, Lingshu-7B, LLaVA-Med).

Results

Hallucination Detection (vs. RadFlag, SE, VASE): - Average +3.0% PR-AUC and +3.9% ROC-AUC over the second-best method across all datasets/models. - Up to +18.5% PR-AUC gain on InternVL3-2B MIMIC-VQA (+10.77% PR-AUC, +8.89% ROC-AUC specifically).

Medical Report Refinement: - Average F1-5 improvement of +7.01% on MIMIC-CXR and +8.16% on IU-Xray. - InternVL-8B: F1-5 from 49.80% → 64.96% (+15.16%) on MIMIC-CXR. - CHAIR-I reduction up to 11.86% (Qwen2-3B, IU-Xray), Micro-F1 improvement up to +12.5%.

Med-VQA: - Mean accuracy gain of +13.04% across datasets. - Qwen2-3B on MIMIC-VQA: 51.83% → 78.99% (+27.16%). - Lingshu-7B on VQA-RAD: 78.09% → 91.24% (+13.15%).

Ablation (MIMIC-CXR): - Textual axis only: F1=0.5986; Visual axis only: F1=0.5460; Both: F1=0.6112; Neither: F1=0.5064.

Limitations

  • Requires a reliable phrase-grounding model to localize claims to bounding boxes; grounding failures propagate as errors.
  • Evaluation limited to chest X-ray datasets (MIMIC-CXR, IU-Xray, VQA-RAD, MIMIC-VQA); generalization to other imaging modalities (CT, MRI, pathology) is undemonstrated.
  • Counterfactual masking adds inference-time overhead proportional to the number of claims per report, which may be prohibitive in real-time clinical settings.
  • Correction relies on an LLM prompt to invert clinical meaning, which may not always produce medically coherent rewrites (not quantified).
  • Code not yet released (pending acceptance).

Relevance to Vision-Language Models

CoEV directly addresses the critical reliability gap in medical VLMs by moving from correlation-based grounding checks (attention maps) to causal counterfactual probing — a methodological shift applicable to any VLM deploying visual grounding. The four-quadrant taxonomy provides a structured decomposition of VLM failure modes that is more informative than binary hallucination labels, relevant to interpretability research across VLM domains. The plug-and-play, training-free design means CoEV can be layered onto existing medical VLMs without architectural changes, lowering the barrier to deployment. For researchers tracking VLMs, this work exemplifies a growing line of post-hoc, evidence-centric evaluation frameworks that complement output-filtering and RLHF-based approaches.