Quantifying Cross-Modality Memorization in Vision-Language Models

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper systematically studies cross-modality memorization in VLMs: whether facts learned in one modality (image or text) transfer to the other. Using a synthetic persona dataset (100 identities, paired images+descriptions), they fine-tune Gemma-3 family models and find that cross-modal transfer occurs but is significantly asymmetric, with image→text transfer outperforming text→image. Neither larger models nor diverse training inputs close this gap.

Problem

Prior memorization research is unimodal—LLM text regurgitation or diffusion model image reproduction. As VLMs become the dominant architecture for knowledge-intensive tasks and personalized assistants, the question of how factual knowledge memorized in one modality propagates across modality boundaries is unaddressed, with direct implications for privacy, unlearning efficacy, and multi-hop reasoning.

Method

The authors construct a synthetic "persona" canary dataset: 100 unique fictional identities, each with 100 image variants (Imagen 3) and 100 textual description variants (Gemini 2.0), sharing a unique name and a surrogate PII field ("favorite number"). Gemma-3 (4B/12B/27B) is fine-tuned with LoRA (rank=32, α=32) exclusively on one modality (image or description) to predict the person's name. Cross-modal transfer is measured by testing on held-out inputs from the opposite modality. Additional experiments probe: (1) training data diversity (single vs. varied inputs per epoch), (2) model scale, (3) in-distribution vs. OOD image-caption augmentation as mitigation, (4) machine unlearning via re-association with random names, and (5) multi-hop reasoning (Desc⇒Name + Name⇒FN ⇒ Desc⇒FN?).

Key Contributions

• First systematic study of cross-modality factual memorization in LLaVA-style VLMs using a controlled synthetic canary dataset.
• Empirical demonstration that cross-modal transfer is asymmetric: image→text transfer is more effective than text→image across all model sizes (4B, 12B, 27B Gemma-3).
• Finding that neither input diversity nor larger model scale improves the cross-modal transferability rate (slope), only within-modality accuracy.
• Identification of a cross-modal "multi-hop curse": models trained on Desc⇒Name and Name⇒FN achieve only 8–11% on Desc⇒FN despite near-perfect single-hop accuracy.
• Cross-modal unlearning asymmetry: unlearning in a different modality than the learning modality fails to suppress recall in the source modality.
• Baseline mitigation: in-distribution synthetic image-caption augmentation meaningfully improves cross-modal transferability; OOD data (COCO) does not.

Results

• Source vs. target accuracy gap: Training on descriptions reaches near-perfect source accuracy; target (image) accuracy lags substantially. Training on images saturates and overfits earlier, with cross-modal text accuracy still significantly lower than source.
• Model scale (Gemma-3 4B → 27B): Higher accuracy in both modalities, but the transferability slope (target/source ratio) remains unchanged across scales.
• Multi-hop: Desc⇒Name + Name⇒FN model achieves 1.00 on each sub-task but only 0.08 on Desc⇒FN. Image⇒Name + Name⇒FN achieves 0.99 on Name⇒FN but 0.05 on Image⇒FN.
• Augmentation mitigation: In-distribution (ID) synthetic pairs consistently raise the target accuracy curve above baseline; OOD (COCO) stays near baseline.
• Unlearning: When unlearning is applied in the same modality as training, accuracy drops are symmetric; cross-modality unlearning shows strongly asymmetric drops (the non-targeted modality is barely affected).

Limitations

• Dataset limited to 100 personas and two modalities (text and image); findings may not generalize to audio, video, or richer VLM settings.
• LoRA fine-tuning on canary data is a controlled stress-test, not representative of pretraining-scale memorization dynamics.
• The "unlearning" method (re-association with random names) is a workaround, not standard gradient ascent; limits comparability with real-world unlearning deployments.
• In-distribution augmentation requires synthetic paired data generation, which may be unavailable or costly in real scenarios.
• All experiments use Gemma-3 family; cross-architecture generalization is not validated.

Relevance to Vision-Language Models

This paper directly probes an understudied property of LLaVA-style VLMs: the degree to which the shared latent space integrates factual knowledge across modalities. The asymmetric image→text vs. text→image transfer suggests the text decoder is more efficiently queried than visual representations, which has implications for how VLMs should be evaluated for both capability (multi-hop reasoning) and safety (privacy leakage across data types). For those tracking VLMs, the cross-modal multi-hop curse and the failure of unlearning to propagate across modalities are concrete failure modes that complicate deployment in personalized assistant settings where models are fine-tuned on heterogeneous user data.