AirflowAttack: Thermal-Airflow Adversarial Perturbations against Infrared Remote-Sensing Vision-Language Models

🕒 Published (v1): 2026-07-07 16:46 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AirflowAttack is the first adversarial attack targeting infrared (IR) remote-sensing VLMs, using physically plausible thermal-airflow turbulence as the perturbation prior. A lightweight generator synthesizes a single input-agnostic universal adversarial perturbation (UAP) optimized on one surrogate CLIP model, achieving 48.5% mean zero-shot scene-classification ASR across five CLIP backbones and cutting VLM scene accuracy by up to 38.2% (relative) without any target-model access.

Problem

Adversarial robustness of IR remote-sensing VLMs is entirely unexamined. RGB-domain attacks cannot transfer to single-channel thermal imagery (which encodes emitted radiation, not reflected light, following Planck's law), while existing physical thermal attacks target unimodal detectors rather than multimodal VLM pipelines. No prior work addresses the intersection of (i) the thermal-IR modality, (ii) the remote-sensing VLM setting, and (iii) physically grounded universal perturbations.

Method

AirflowAttack parameterizes the UAP \(\delta\) through a lightweight generator \(G_\theta\) that maps a low-dimensional latent code \(z \in \mathbb{R}^{32}\) to a full-resolution single-channel perturbation via transposed convolutions with residual connections (residual ratio \(r=0.60\)). The adversarial image is formed as:

\[x_{adv} = \text{clip}\!\left(x + G \odot \bigl(\epsilon \cdot \tanh(a \cdot P_{air}) + 0.6 \cdot \text{Blur}(\text{Upsample}(Z))\bigr)\right)\]

under \(\|\delta\|_\infty \leq \varepsilon = 100\). Optimization minimizes:

\[\mathcal{L} = \alpha \mathcal{L}_{\text{conf}} + \beta \mathcal{L}_{\text{air}}, \quad \alpha=8,\; \beta=2\]

where \(\mathcal{L}_{\text{conf}}\) is a contrastive image–text alignment loss on the surrogate CLIP model that pushes the adversarial image embedding away from the correct caption:

\[\mathcal{L}_{\text{conf}} = -\log \frac{\exp(\tau \cdot \text{sim}(f_I(x^{adv}), f_T(t^+)))}{\sum_{t \in \{t^+\} \cup \mathcal{T}^-} \exp(\tau \cdot \text{sim}(f_I(x^{adv}), f_T(t)))}\]

and \(\mathcal{L}_{\text{air}} = 1 - \text{Corr}(\delta, \mathcal{A})\) is a spatial Pearson correlation loss against a bank \(\mathcal{A}\) of synthetic airflow templates generated via randomized heat-kernel convolution. The joint optimization runs for 800 steps at \(\eta=0.055\) over both generator parameters \(\theta\) and latent code \(z\). The final UAP transfers to target models without any gradient access.

Key Contributions

• First adversarial attack designed for IR remote-sensing VLMs, introducing thermal-airflow turbulence as an adversarial perturbation prior.
• Lightweight generative parameterization that enforces spatial coherence and physical plausibility via the \(\mathcal{L}_{\text{air}}\) term without sacrificing ASR.
• A single surrogate-optimized UAP achieves 48.5% mean ASR across five CLIP backbones (vs. 27.7–37.0% for IR-specific baselines), with cross-model retrieval flip rates of 94.4–98.8%.
• Benchmark spanning eleven models and four tasks (zero-shot classification, captioning, VQA scene, object F1, IR-cue accuracy) on a 10,000-sample filtered IR dataset.
• Discovery of a paradoxical effect: the attack increases IR-cue confidence in some VLMs, causing hallucinated thermal explanations (temperature gradients, convection) for the perturbation.

Results

• CLIP zero-shot ASR (mean): AirflowAttack 48.5% vs. atmospheric turbulence 33.6%, stripe noise 37.0%, hot/cold blocks 29.2%, sensor drift 27.7%.
• Per-backbone ASR: OpenAI-CLIP-B32 54.2% (surrogate), GeoRSCLIP-B32 51.7%, RemoteCLIP-B32 50.6%, OpenCLIP-B32 47.9%, OpenAI-CLIP-L14 38.3%.
• Cross-model retrieval flip rate: 94.4% (OpenAI-CLIP-L14) to 98.8% (GeoRSCLIP-B32) with no target-model access.
• VLM scene classification drops: Qwen2.5-VL-7B 35.24→21.83 (−38.1%), InstructBLIP 23.66→14.63 (−38.2%), LLaVA-1.6 23.54→16.22 (−31.1%).
• ROUGE-L captioning: AirflowAttack achieves lowest ROUGE-L on Qwen2.5-VL (11.70), InstructBLIP (11.91), and GeoChat (9.80) among all methods.
• IR-specific domain-adapted models (RemoteCLIP, GeoRSCLIP) show the largest performance gap vs. baselines, suggesting domain pretraining amplifies sensitivity to physically structured thermal perturbations.

Limitations

• Gray-box setting requires access to a surrogate CLIP model; fully black-box (no-surrogate) scenarios are not evaluated.
• \(\varepsilon = 100\) (out of 255) is a large \(L_\infty\) budget, raising questions about human detectability in high-stakes applications.
• Physical realizability in actual deployment (e.g., using real thermal emitters in the field) is not empirically demonstrated — the attack remains digital.
• ROUGE-L is shown to understate captioning degradation; semantic metrics (e.g., BERTScore, CLIPScore) are not reported for captioning.
• Evaluation is on a filtered subset of public RS datasets; out-of-distribution or finer-grained scene categories are not explored.
• Object F1 and IR-cue accuracy results under AirflowAttack vs. baselines are partially truncated in the provided text.

Relevance to Vision-Language Models

AirflowAttack directly probes the robustness of the CLIP-family and instruction-tuned VLM pipeline in a safety-critical non-RGB modality, revealing that universal transferable perturbations optimized on one surrogate generalize broadly across architecturally distinct VLMs (LLaVA, InstructBLIP, Qwen2.5-VL, GeoChat). The paradox of increased IR-cue confidence under attack is particularly significant for the VLM robustness literature, as it suggests that physically structured perturbations can exploit — rather than merely disrupt — domain-specific multimodal feature representations, causing confident hallucination rather than mere accuracy degradation. This work extends the nascent literature on universal adversarial examples for VLMs (AdvCLIP, Co-Attack) to a physically grounded, modality-specific regime, establishing that the standard assumption of RGB-domain threat models does not transfer cleanly to IR-VLM deployments.