Skip to content

Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

🕒 Published (v1): 2026-07-20 23:19 UTC · Source: Arxiv · link

Why this paper was selected

Causal audit proves attention heatmaps don't reflect actual medical VLM evidence

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Attention and saliency heatmaps produced by medical VLMs on chest X-rays are visually reassuring but causally unfaithful: no evaluated VLM satisfies both conditions for a grounded explanation (image use + attention-causal alignment). CXR-trained CNN classifiers pass the same three-axis protocol, confirming the failure is model-specific, not metric-specific.

Problem

Radiology VLMs routinely surface attention heatmaps as explanations, yet prior work has not tested causally whether these heatmaps highlight the image regions that actually drive the prediction. Visual inspection of a "hot" region cannot distinguish genuine grounding from coincidental overlap; the faithfulness of VLM attention on chest X-rays was unaudited across multiple attribution methods, controlled baselines, and causal perturbation.

Method

A three-axis evaluation protocol is applied to eight model configurations across six families:

  1. Grounding overlap (PadChest, n=637): Last-text-token attention over 256 image tokens (averaged over heads, layers 10–20, thresholded at 90th percentile) is scored against radiologist bounding boxes versus three controls (shifted, random-far, random-size).
  2. Pixel-mask attribution (CheXlocalize, n=643): Four saliency methods (raw attention, attention rollout, gradient×attention, Winsor-CAM), plus integrated gradients and RISE, are upsampled to 224×224 and evaluated by pointing-game hit rate, attribution mass in mask (AMiM), and pixel-AP with paired permutation tests.
  3. Counterfactual causal maps: Each of 256 16×16 patches is mean-filled independently; margin shift \(\delta_{r,c} = m_{\text{orig}} - m_{\text{occluded}}\) is recorded, yielding a causal importance map. Spearman \(\rho\) between the saliency map and causal map is the primary alignment metric. ROI-delete vs. random-region occlusion is used for CheXagent-2-3b, which does not expose cross-modal attention.

Models: MedGemma-4B (base, targeted LoRA layers 15–19, full LoRA all 34 layers), LLaVA-RAD-7B, Qwen3-VL-8B-Instruct, CheXagent-2-3b; DenseNet121-Chex and ResNet50-All serve as positive controls via Grad-CAM. Secondary replication on VinDr-CXR (n=300).

Key Contributions

  • Demonstrates causally (via patch occlusion) that attention anti-correlates with causal importance for all MedGemma variants (\(\rho\): −0.098 base, −0.050 targeted LoRA, −0.168 full LoRA) and Qwen3-VL (\(\rho = -0.117\)); LLaVA-RAD's positive \(\rho = +0.137\) is shown to arise from near-zero image use (99.1% text-only agreement, causal AMiM 0.005).
  • Establishes that the failure is method-agnostic: raw attention, rollout, gradient×attention, integrated gradients, and RISE all anti-correlate with causal importance (\(\rho\) −0.18 to −0.28); only RISE and Winsor-CAM are near zero, not positive.
  • Shows the same protocol detects genuine localization in CXR classifiers (\(p < 10^{-4}\) on every metric), ruling out a metric or modality artifact.
  • Documents that consistency-improving fine-tuning (reducing paraphrase flips) does not improve causal grounding; raw-attention AMiM still leaves >80% of mass outside the radiologist mask.
  • Finds specialist VLM grounding is annotation-style-dependent: CheXagent is causally grounded on VinDr-CXR (\(\Delta = +0.568\)) but not on PadChest (CI includes zero).

Results

  • Bounding-box overlap (PadChest): MedGemma base true-box coverage 0.037, below shifted (0.042), random-far (0.052), and random-size (0.048) controls; LLaVA-RAD true-box coverage 1.2%.
  • AMiM (CheXlocalize): No VLM exceeds 22% attribution mass inside the radiologist mask under any method; best is full-LoRA raw attention at AMiM = 0.214.
  • Pointing game: Gradient×attention reaches 44–51% for MedGemma variants; Qwen3-VL raw attention achieves exactly 0%.
  • Attention-causal Spearman \(\rho\) (PadChest, signed causal map): −0.098 (base), −0.050 (targeted LoRA 500), −0.168 (full LoRA 500), −0.145 (full LoRA 12K); Qwen3-VL −0.117; all 95% bootstrap CIs exclude zero.
  • Per-method \(\rho\) vs. absolute causal shift: Raw attention −0.263/−0.202/−0.253; attention rollout −0.257/−0.178/−0.238; gradient×attention −0.243/−0.189/−0.258; integrated gradients −0.276/−0.208/−0.281 (base/targeted/full LoRA).
  • CXR classifiers (positive controls): DenseNet insertion AUC 0.690 vs. random 0.666 (\(p < 10^{-4}\)); all VLMs \(|\Delta\text{AUC}| \leq 0.024\).
  • VinDr-CXR replication: Attention-causal \(\rho\) near zero for all MedGemma variants (+0.053 to +0.061); true-box AMiM exceeds shuffled in only 27–33% of cases.

Limitations

  • Causal maps are computed on a 16×16 token grid; sub-patch pathologies may be partially masked and localization of fine-grained findings is a lower bound.
  • Qwen3-VL is evaluated at a native 8×8 grid (memory constraint), reducing spatial resolution of the causal map.
  • Mid-gray patch fill is used throughout; although a fill-sensitivity check finds no positive \(\rho\) under any fill, the specific fill choice remains a protocol dependency.
  • CheXagent's lack of accessible cross-modal attention restricts its evaluation to the margin-only ROI test; full multi-method analysis was not possible.
  • Only eight model configurations across six families are tested; architectures with explicit grounding heads (e.g., MIMO) are not evaluated.
  • The positive-control CXR classifiers are not claimed to be clinically superior overall; the contrast is specific to the localization protocol.

Relevance to Vision-Language Models

This paper delivers the most comprehensive causal faithfulness audit of medical VLM explanations to date, directly challenging the assumption that attention heatmaps in multimodal models reflect the image evidence driving a prediction. The finding that attention anti-correlates with causal importance—and that this holds across six attribution methods including axiomatic (integrated gradients) and model-agnostic (RISE) approaches—is a strong negative result with broad implications for any VLM deployed in high-stakes visual question answering. For researchers working on grounding, interpretability, or clinical deployment of VLMs, the three-axis protocol (controlled overlap + pixel-mask attribution + patch-occlusion causal maps) provides a reusable evaluation template. The result also intersects ongoing debates about modality utilization in VLMs: the "consistency without grounding" finding shows that reducing language-side noise in a VLM does not force the model to attend to visually causal regions, a nuance relevant to fine-tuning and RLHF strategies for VLMs.