Skip to content

Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs

🕒 Published (v1): 2026-07-17 16:40 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces a lightweight, gradient-free saliency method for ViT-LLM architectures that maps language model attention over visual patch tokens back onto the original chart image, producing per-output-token saliency maps during free-form answer generation. Deletion-based evaluation on VLAT confirms the maps are causally faithful, achieving AUC of 0.020 versus 0.070 for AG-CAM. The method requires no gradient computation, no architectural modification, and no extra inference pass.

Problem

Existing chart question answering (CQA) benchmarks measure only textual correctness, leaving unaddressed whether VLMs attend to the correct visual evidence — a model can achieve high accuracy via parametric memory shortcuts while ignoring the chart entirely. Existing saliency methods (Grad-CAM, AG-CAM) were designed for CNNs or classification tasks and operate over input tokens rather than generated output tokens, making them ill-suited for open-ended text generation in ViT-LLM architectures.

Method

For each generated output token \(y_t\), the method aggregates the LLM's cross-attention weights over the \(N_p\) visual prefix tokens across all \(N_h\) heads and \(N_l\) layers:

\[\bar{A}^{(t,l)} = \frac{1}{N_h}\sum_{k=1}^{N_h} A^{(t,l,k)}, \qquad A^{(t)}_{\text{global}} = \frac{1}{N_l}\sum_{l=1}^{N_l} \bar{A}^{(t,l)}\]

The row at query position \(r_t = T + t - 1\) is extracted, re-normalized over the visual prefix positions to give \(\tilde{w}_t \in \mathbb{R}^{N_p}\), reshaped into a \(G \times G\) coarse grid, max-normalized, and bilinearly upsampled to full image resolution to yield the per-token pixel-wise saliency map \(S_t(x,y)\). Aggregate maps for faithfulness evaluation are formed by L1-normalizing each token's distribution and averaging over all generated tokens. No backpropagation is required.

Key Contributions

  • Token-level, attention-guided saliency extraction tailored to ViT-LLM architectures, operating over generated output tokens rather than input tokens.
  • Gradient-free and architecture-unmodified: runs within a single standard forward/generation pass.
  • Deletion-based faithfulness evaluation on VLAT (13 correctly answered samples with ChartGemma), comparing against AG-CAM and random deletion baselines.
  • Extended visualization literacy evaluation: full VLAT results for ChartGemma (34.0%) and LLaVA-1.5 (22.6%) in open-ended generation mode.

Results

  • Faithfulness (AUC, lower is better): proposed method = 0.020; AG-CAM = 0.070; random = substantially higher (serves as upper-bound control). Both saliency methods outperform random, but the proposed method identifies causally critical pixels more precisely.
  • Accuracy on Mini-VLAT: ChartGemma 58.3% (replicates [6] exactly); LLaVA-1.5 25% (vs. 20% in [6], within stochastic tolerance).
  • Accuracy on full VLAT (open-ended): ChartGemma 34.0%; LLaVA-1.5 22.6%.
  • Deletion curves show performance collapses almost entirely after removing a very small fraction of top-ranked pixels, indicating chart QA accuracy is spatially concentrated on specific axis labels or marks.

Limitations

  • Deletion evaluation uses only 13 samples (ChartGemma on VLAT correctly-answered, non-True/False questions), limiting statistical power.
  • LLaVA-1.5 is excluded from deletion evaluation due to insufficient correct answers; generalizability across model families is thus unverified.
  • Mean aggregation across layers and heads is used instead of more sophisticated rollout strategies; the paper acknowledges rollout alternatives exist but does not compare them.
  • Bilinear upsampling from coarse patch resolution (\(G \times G\)) to full image resolution smooths spatial precision; sub-patch localization is not possible.
  • Evaluation is restricted to two models and one domain (data visualizations / chart QA); broader generalization to other VLM tasks is not demonstrated.

Relevance to Vision-Language Models

This work directly addresses the interpretability of ViT-LLM architectures — the dominant VLM paradigm (LLaVA, ChartGemma) — by providing a saliency diagnostic that is faithful to the generative attention mechanism rather than borrowed from classification-era CNN methods. The finding that VLMs can answer chart questions correctly while potentially relying on spurious shortcuts (confirmed by prior work cited) motivates this class of faithfulness probing, relevant to anyone studying VLM visual grounding and shortcut learning. The method's output-token-centric framing (saliency per generated token) is a useful conceptual shift for the VLM interpretability community, distinct from input-attribution approaches. Practically, the zero-overhead design makes it deployable alongside any ViT-LLM inference pipeline as a real-time diagnostic.