Skip to content

Chameleon: Adaptive Adversarial Agents for Scaling-Based Visual Prompt Injection in Multimodal AI Systems

🕒 Published (v1): 2025-12-04 15:22 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Chameleon is an adaptive adversarial framework that exploits image downscaling—a ubiquitous VLM preprocessing step—to inject hidden malicious prompts that are invisible at full resolution but activate after scaling. Unlike static prior attacks, Chameleon uses a closed-loop, reward-driven optimization that iteratively refines pixel perturbations based on live VLM feedback, achieving 87–91% attack success rates against Gemini 2.5 Flash.

Problem

Existing scaling-based visual prompt injection attacks are static and single-shot: they craft adversarial images once with no feedback, making them brittle and model-specific. No prior work combined adaptive feedback loops with image-scaling exploits, and none studied the downstream impact on multi-step agentic pipelines where compromised intermediate decisions propagate.

Method

Chameleon constructs an adversarial image \(I_{adv} = \text{clip}(I + \delta, 0, 255)\) where \(\delta\) is initialized from \(\mathcal{U}(-\epsilon, +\epsilon)\). After each VLM query, a scalar reward

\[R = w_1 \cdot s - w_2 \cdot d - w_3 \cdot (1 - c)\]

aggregates attack success \(s\), normalized visual distance \(d\), and model confidence \(c\) (with empirically tuned weights \(w_1=10, w_2=0.5, w_3=0.2\)). Two optimizers refine \(\delta\) iteratively: (1) hill-climbing, which accepts updates only when reward strictly improves; and (2) a genetic algorithm (GA) that maintains a population of 20 perturbations, applying crossover \(\delta_{offspring} = \lambda\delta_1 + (1-\lambda)\delta_2 + \mathcal{N}(0,\sigma^2)\). The loop runs until \(s=1\) or 50 iterations are exhausted. Attacks are evaluated against Gemini 2.5 Flash via its public API using 4368Ă—4368 source images downsampled with bicubic, bilinear, and nearest-neighbor interpolation.

Key Contributions

  • Chameleon framework: first adaptive, feedback-driven scaling-based prompt injection attack against production VLMs.
  • Dual optimizer comparison: hill-climbing vs. GA trade-off analysis across ASR, visual imperceptibility, convergence speed, and API cost.
  • Agentic pipeline evaluation: demonstrates >45% reduction in multi-step decision-making accuracy under attack.
  • Defense proposal: multi-scale consistency checks flagged as necessary mitigations.

Results

  • ASR: Hill-climbing 87%, GA 91% (vs. static baseline average 32.1%).
  • Visual distance (normalized \(\ell_2\)): Hill-climbing mean 0.0847, GA mean 0.0693; both well below perceptual detection thresholds.
  • Convergence: Hill-climbing converges in 23.4 mean iterations (127.3 sec); GA requires 31.7 iterations (189.6 sec).
  • API efficiency: Hill-climbing 12.47 calls/trial, GA 15.84 calls/trial—both within free-tier rate limits.
  • Decision Manipulation Rate (DMR): 87% (HC) / 91% (GA); model confidence drops 0.18–0.21 post-injection.
  • Prompt generalization: ASR 84–93% across five semantically distinct prompts; 86–92% across all three interpolation methods.
  • Agentic impact: decision accuracy reduced by >45% in multi-step tasks.

Limitations

  • Evaluated on a single VLM architecture (Gemini 2.5 Flash); cross-model generalization untested.
  • Dataset is only 20 images; may not capture out-of-distribution or high-semantic-complexity content at scale.
  • Optimization reward weights \(w_1, w_2, w_3\) are empirically chosen without principled tuning.
  • No evaluation of proposed defenses (multi-scale consistency checks) against Chameleon itself.
  • Black-box API access assumed; white-box or gradient-based baselines not compared.

Relevance to Harnesses / Meta-Harnesses

Chameleon's core attack mechanism is itself a meta-harness: it wraps a target VLM in an outer optimization loop that treats the model as a black-box oracle, iteratively querying it and adapting its strategy based on structured feedback—precisely the pattern used in agentic meta-harnesses for orchestration and self-refinement. The work directly demonstrates that production agentic pipelines (multi-step reasoning chains) are disproportionately vulnerable when any component in the preprocessing-to-inference chain is exploitable, because errors compound across steps. For harness designers, this highlights a critical threat model: any harness that pipes external images through standard resize operations before VLM inference inherits this attack surface without additional hardening.