Skip to content

Attention! Your Vision Language Model Could Be Maliciously Manipulated

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VMA (Vision-language model Manipulation Attack) is a white-box adversarial attack that uses Adam-style first- and second-order momentum optimization with a differentiable sigmoid reparameterization to craft imperceptible image perturbations that precisely control every output token of a VLM. The authors prove theoretically that visual perturbations have a smaller certified robustness radius than textual perturbations, making images a more potent attack surface. A single unified framework enables jailbreaking, hijacking, hallucination induction, privacy extraction, DoS, sponge examples, and watermarking.

Problem

Existing VLM adversarial attacks either append adversarial text suffixes or embed typography into images, targeting only coarse behaviors (e.g., triggering an affirmative prefix for jailbreaking). No prior work demonstrated that imperceptible pixel-space perturbations can precisely control every token in a VLM's autoregressive output sequence. PGD, the standard image-domain optimizer, fails on this task due to gradient discontinuities from the projection step and poor convergence on the deep, iteratively computed conditional probability objective.

Method

VMA frames output manipulation as maximizing P(t_{n+1:n+l} | [E_T(t_{1:n}), E_I(x_adv)]) subject to an ℓ_∞ perturbation budget ε. Two modifications over PGD address optimization instability:

  1. Differentiable transformation: replaces the non-differentiable ℓ_∞ projection with x_adv = x̲ + σ(z)·(x̄ − x̲), where σ is sigmoid and x̲, x̄ are pixel-space bounds. Optimization is over z ∈ ℝ (unconstrained), making the objective differentiable everywhere.
  2. Adam momentum: first- and second-order moment estimates (β₁=0.9 default) replace the PGD sign-gradient step, improving convergence stability on the deep, recursive loss landscape.

The attack targets arbitrary desired token sequences; task-specific applications (jailbreaking, DoS, etc.) are obtained by selecting which tokens to enforce (initial affirmative tokens, EOS suppression, misinformation strings, etc.).

Key Contributions

  • Theoretical proof (via certified radius comparison) that visual perturbations induce strictly smaller certified robustness radii than textual synonym substitutions (r₁^img < r_text), formally explaining why images are a more dangerous attack surface.
  • VMA algorithm: first method demonstrated to achieve token-precise, full-sequence adversarial control of VLMs via imperceptible image perturbations.
  • Unification of eight distinct attack/protection scenarios under one framework without task-specific redesign.
  • Ablation isolating the contributions of differentiable transformation (+8 pp ASR, 30% fewer iterations) and momentum (+17 pp ASR, 57% fewer iterations) individually and jointly.

Results

  • Manipulation (output control): average ASR 90.25% across LLaVA, Qwen2-VL, Phi3, DeepSeek-VL at ε=4/255; individual runs reach 100% (Qwen2-VL, prompt 2).
  • Jailbreaking (AdvBench / MM-SafetyBench): ≥88.05% / ≥80.84% ASR at ε=4/255 across all four models; clean-image baselines are ≤42.77% / ≤40.02%.
  • Hijacking (VQA-CLS, VQA-CPT, CLS-CPT on COCO): average ~90% ASR at ε=16/255; baseline ~36% (natural ambiguity).
  • Hallucination (POPE benchmark): ≥98.97% ASR at ε=4/255 vs. ≤6.69% baseline; cross-prompt transferability observed within topic.
  • Privacy breaches (MLLMU-Bench): ≥93.38% ASR on LLaVA/Qwen2-VL/Phi3 at ε=4/255; 99.54% on DeepSeek-VL at ε=8/255.
  • DoS: ≥88.20% ASR at ε=4/255 on MM-Vet non-rejectable pairs.
  • Sponge examples: >200× inference time increase on DeepSeek-VL, >400× on others at ε=4/255; ASR >70% for hitting 10,000-token limit.
  • Watermarking (874 Van Gogh paintings): ≥78.29% protection rate at ε=4/255; ≥93.14% at ε=16/255.

Limitations

  • Poor cross-architecture transferability; effectiveness is primarily demonstrated in white-box settings.
  • Attack difficulty scales with target output sequence length; very long target sequences reduce effectiveness.
  • Momentum increases per-iteration runtime (though net convergence is faster).
  • No defenses evaluated; robustness of existing VLM safety alignment to this attack class is not studied from a defender's perspective.
  • Restricted to open-source models with accessible gradients; black-box applicability is future work.

Relevance to Vision-Language Models

VMA directly exposes a fundamental security gap in VLMs: the continuous, high-dimensional nature of the image modality makes it a disproportionately powerful attack surface relative to discrete text, a claim the paper formalizes with certified-radius theory. For researchers tracking VLMs, this is a significant safety finding because it shows that current safety alignment (RLHF, refusal fine-tuning) is wholly insufficient against image-space adversarial perturbations — bypassing safeguards does not require jailbreak prompts or visible image edits. The dual-use watermarking application also opens a direction for using adversarial image perturbations as a copyright protection mechanism for VLM-consumed content, relevant to ongoing work on model governance and output attribution.