Skip to content

VLMs can Aggregate Scattered Training Patches

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VLMs can integrate visual information scattered across multiple training samples sharing the same text descriptions—a capability the authors term "visual stitching." This emergent cross-sample reasoning enables a practical data poisoning attack: splitting harmful images into benign-looking patches that evade content moderation, then finetuning VLMs on the filtered patches to internalize dangerous associations.

Problem

Data moderation removes harmful (image, text) training pairs, but an adversary can split a harmful image into small patches—each appearing individually benign—pair each patch with the same (potentially misleading) text, and distribute them across many training samples. Existing moderation tools (OpenAI Moderation API, ShieldGemma-2) operate per-sample and cannot detect content that is harmful only in aggregate, leaving VLMs exposed to covert knowledge injection.

Method

Visual stitching evaluation: Three synthetic datasets (food, animal, landmark; 20 images each with unique synthetic IDs) are constructed. Each (image, ID) pair is split into {(patch, ID)} pairs at split factors f ∈ {1, 2, 4, 8} (i.e., f² patches per image). VLMs (Qwen2-VL, Qwen2.5-VL, Gemma-3, Llama-3.2-Vision, InternVL3, LLaVA-1.5/1.6; up to ~100B params) are finetuned on patch-text pairs and evaluated on image-based stitching (full image → correct ID) and reference-based stitching (text reference → correct ID). Performance is measured by mean rank of the correct ID among all candidates (lower = better).

Adversarial attack simulation: 20 harmful images (10 sex-related, 10 violence-related) are split into patches with f ∈ {2, 4, 8}, paired with adversarially chosen "safe"/"unsafe" labels. Patches flagged by the OpenAI Moderation API are discarded; VLMs are finetuned on remaining patches. Stitching success is measured by the VLM's mean rank for the adversarial label on full harmful images or text references to them.

Robustness ablations: Visual stitching is tested under data mixture (poisoned patches mixed with up to 20,000 SFT samples, representing as little as 0.4% of the corpus) and label noise (10–40% random label corruption).

Key Contributions

  • Introduces and formalizes visual stitching—a VLM capability to aggregate visual information distributed across training samples linked by shared text descriptions.
  • Provides three synthetic benchmarks (food, animal, landmark) and a dual evaluation protocol (image-based and reference-based stitching) using mean rank as the metric.
  • Demonstrates empirically that most open-source VLMs (~7–12B params) exhibit strong image-based and non-trivial reference-based visual stitching across multiple model families.
  • Shows that visual stitching enables a practical moderation-evasion data poisoning attack: with 8×8 splits, only 9% of patches from harmful images are flagged by the OpenAI Moderation API, yet finetuning on remaining patches causes VLMs to misclassify harmful content.
  • Establishes robustness of visual stitching under data mixture and label noise conditions.

Results

  • Image-based stitching: All tested ~10B models achieve near-zero mean rank even at f=8 (random baseline ~9.5); worst case (Gemma-3-12B on food, f=8) still achieves rank < 3.
  • Reference-based stitching: Substantially harder; Qwen2-VL-7B and Qwen2.5-VL-7B-Instruct are consistently best, maintaining meaningful stitching at f=8. Most other models approach random performance at f=8 on food/animal.
  • Patch splitting f=2 outperforms f=1 (full image) on reference-based stitching for most models, suggesting large-patch splitting acts as visual data augmentation.
  • Medium-sized models (~7–12B) outperform both small and large variants; Qwen2-VL-2B and InternVL-1B fail reference-based stitching; very large models overfit and generalize less.
  • Moderation evasion: At f=8, OpenAI Moderation API flags only 9% of patches (17/1280 for some images); ShieldGemma-2 flags 19/20 full images but misses most patches.
  • Adversarial finetuning: After filtering via OpenAI Moderation API, Qwen2-VL-7B and others still achieve above-chance ranks for adversarial labels on both full harmful images and text references.
  • Data mixture: Visual stitching persists even when poisoned patches constitute ~0.4% of the corpus (20 images × 4 patches / 20,000 SFT samples); SFT mixing slightly improves reference-based stitching.
  • Label noise: Image-based stitching is robust through 40% label corruption; reference-based stitching degrades at 40% noise, especially with f=8.

Limitations

  • Only open-source VLMs are evaluated; proprietary models (GPT-4o, Gemini) may differ substantially.
  • Visual stitching—especially reference-based—is highly unstable across training runs: performance fluctuates significantly with learning rate and random seed, and correct IDs are rarely the top greedy-decoded output (only ranking improvements in log-probabilities are observed).
  • The adversarial attack is a proof-of-concept with 20 harmful images, not a comprehensive attack framework with realistic scale or diversity.
  • Evaluation datasets are small (20 images per domain) with synthetic IDs; generalization to natural text descriptions and larger corpora is not fully established.
  • Mechanistic understanding of why visual stitching emerges (e.g., role of M-RoPE vs. dynamic resolution in Qwen2) remains hypothetical and uninvestigated.
  • Adding positional context for patches consistently hurts visual stitching (ablation in Appendix A.4), with no explanation offered for this counterintuitive finding.

Relevance to Vision-Language Models

This paper exposes a previously uncharacterized generalization mechanism in VLMs—cross-sample visual aggregation—that is distinct from in-context reasoning and connects to the broader literature on out-of-context reasoning in LLMs. For VLM safety researchers, it demonstrates a concrete failure mode of sample-level data moderation that scales poorly with model capability, since more capable models (especially those with M-RoPE and dynamic resolution, e.g., Qwen2-VL) exhibit stronger stitching. It also raises fundamental questions about what knowledge VLMs acquire during finetuning on large-scale web data, since moderation pipelines cannot currently detect contextually harmful content that only emerges from aggregation across training samples.