Skip to content

Interpretability Transfer from Language to Vision via Sparse Autoencoders

🕒 Published (v1): 2026-05-24 08:47 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; sparse autoencoders transfer language interpretability to vision

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VISTA (Visual Interpretability via SAE Transfer Alignment) transfers interpretability from language to vision in LLaVA-style VLMs by constraining the visual projector to map visual tokens into a frozen LLM's pre-existing, labeled textual SAE feature space. This eliminates the need to train vision-specific SAEs or re-label features. The approach achieves a threefold increase in concept matching rate and enables causally validated, localized visual steering.

Problem

Sparse autoencoders (SAEs) have successfully decomposed polysemantic LLM activations into interpretable, labeled concepts, but applying them to visual modalities is blocked by two failure modes: (1) vision-only SAEs produce ambiguous, hard-to-label features and cannot explain how visual inputs are processed once they enter the LLM residual stream; (2) fine-tuning existing LLM SAEs (e.g., Gemma Scope) to incorporate vision requires ~20% of GPT-3-scale compute and hundreds of thousands of re-labeling calls. Neither approach reveals cross-modal interpretability in a tractable way.

Method

VISTA augments the standard LLaVA training objective with an SAE Reconstruction Loss. A frozen vision encoder produces \(N\) visual tokens \(v = V(I) \in \mathbb{R}^{N \times D_v}\); a learnable linear projector \(P_\theta\) maps them to the LLM embedding space \(e_v = P_\theta(v) \in \mathbb{R}^{N \times D_\text{llm}}\). The projector is trained to minimize:

\[\mathcal{L}_\text{total} = \mathcal{L}_\text{CE} + \mathcal{L}_\text{SAE}\]

where \(\mathcal{L}_\text{SAE} = \frac{1}{|M|} \sum_{l \in M} \|\text{SAE}_l(e^l_v) - e^l_v\|^2_2\) enforces that visual token hidden states at selected LLM layers \(M\) are reconstructible by the frozen text SAE. The LLM and vision encoder remain frozen throughout. Once aligned, the top-activating SAE latents per visual token are labeled via Neuronpedia. Localized steering performs arithmetic on specific patch vectors: \(v_\text{steered} = v_\text{source} - \beta \cdot v_\text{remove}\) (removal) or \(v_\text{source} + \alpha \cdot v_\text{add}\) (replacement), leaving surrounding tokens unchanged.

Key Contributions

  • Training objective that aligns visual tokens to the text SAE manifold without training or fine-tuning vision SAEs.
  • Empirical finding that "delayed alignment" (visual tokens converging to the text SAE manifold only in deep layers) is architecture-dependent: it occurs in Gemma but not LLaMA-3.1-8B, where unconstrained visual tokens never converge; VISTA forces immediate convergence in both.
  • Rigorous encoder comparison via Spatial Localization Accuracy (SLA): DINOv2 (SLA=0.927) substantially outperforms I-JEPA (0.702) and CLIP (0.201) for patch-level concept localization under Gemma-2-2B-it.
  • Causal validation through localized concept steering: 35% improvement in object removal and 47% in object replacement over best vision-only SAE baselines.
  • Out-of-distribution generalization confirmed on DomainNet, showing alignment is structural rather than a training-data artifact.

Results

  • Matching rate: VISTA achieves a >3× increase in LLM-verified concept matching rate (DINOv2 + Gemma-2-2B-it) compared to unconstrained baseline across layers.
  • Spatial Localization Accuracy (Gemma-2-2B-it backbone):
  • DINOv2: 0.927
  • I-JEPA: 0.702
  • CLIP: 0.201
  • Visual steering over best vision-only baselines:
  • Object removal: +35%
  • Object replacement: +47%
  • Standard VQA benchmarks (MME, GQA, POPE, LLaVA-Bench): slight regression on Gemma-2-2B-it with SAE constraints; mixed/positive effects on LLaMA-3.1-8B-Instruct (SAE constraint acts as beneficial regularizer); encoder-dependent behavior on Gemma-2-9B-it.
  • OOD: Matching rate on DomainNet (500 images) comparable to in-distribution performance.

Limitations

  • Slight performance regression on standard VQA benchmarks for the smaller Gemma-2-2B-it model, indicating an interpretability–performance trade-off.
  • Relies on pre-existing labeled text SAEs (e.g., from Neuronpedia); quality of visual interpretability is bounded by the completeness and correctness of those labels.
  • Localized steering and interpretability depend heavily on the choice of vision encoder—CLIP's spatial confusion makes it unsuitable for fine-grained interventions regardless of the LLM backbone.
  • The number of SAE-constrained layers requires empirical tuning per architecture (performance-interpretability trade-off analysis).
  • Experiments confined to LLaVA-paradigm architectures; generalization to other VLM families (e.g., encoder-free or cross-attention-based models) is not demonstrated.

Relevance to Vision-Language Models

VISTA directly addresses a core interpretability gap in LLaVA-style VLMs: how visual tokens are represented and processed inside the frozen LLM backbone. By forcing visual projectors to produce tokens that lie on the text SAE manifold, VISTA makes the internal cross-modal alignment tractable to inspect and manipulate—capabilities previously unavailable without bespoke vision SAEs. The finding that DINOv2 vastly outperforms CLIP in patch-level localization is practically significant for VLM design, suggesting self-supervised local-discriminative pretraining produces better visual tokens for fine-grained language grounding. The localized steering mechanism opens a path toward mechanistic hallucination analysis and targeted behavioral control in deployed VLMs without architectural changes.