BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

BadVLA is the first backdoor attack framework targeting Vision-Language-Action (VLA) models, exploiting the Training-as-a-Service (TaaS) paradigm. It uses a two-stage objective-decoupled optimization: first injecting a trigger into the perception module via contrastive feature separation, then fine-tuning the remaining modules on clean data to preserve normal task performance. The attack achieves near-100% attack success rates (ASR) across multiple VLA benchmarks while maintaining clean-task accuracy and resisting common defenses.

Problem¶

Existing backdoor and data-poisoning attacks designed for unimodal systems fail in VLA settings due to three obstacles: (1) long-horizon sequential dynamics that dilute single-step perturbations, (2) cross-modal entanglement between vision, language, and action that prevents unilateral manipulation, and (3) the difficulty of curating poisoned multimodal data at scale. No prior work had systematically investigated backdoor vulnerabilities in end-to-end VLA models, despite their increasing deployment in safety-critical robotics and the openness of large-scale pretrained VLA checkpoints under TaaS.

Method¶

BadVLA decomposes a VLA model into a perception module \(f_p\), backbone \(f_b\), and action head \(f_a\), then applies two sequential training stages:

Stage I – Trigger Injection via Reference-Aligned Optimization. Only \(\theta_p\) is trainable. A frozen reference model \(f_\text{ref}\) provides stable clean-input embeddings. The loss \(\mathcal{L}_\text{trig}\) enforces two simultaneous objectives: (a) cosine/L2 alignment between \(f_\theta(x_i)\) and \(f_\text{ref}(x_i)\) to preserve clean semantics ("Restrict" term), and (b) maximized L2 distance between \(f_\theta(T(x_i,\delta))\) and \(f_\theta(x_i)\) to push triggered representations into an orthogonal subspace ("Trigger Separation" term, weighted by \(\alpha\)). The trigger \(\delta\) is a small visual patch learned jointly with \(\theta_p\).

Stage II – Clean Task Enhancement. \(\theta_p\) is frozen (locking in the latent separation), and \(\theta_b, \theta_a\) are fine-tuned on clean data via standard negative log-likelihood over autoregressive action decoding. Because the backbone and action head see only clean-aligned features during training, triggered inputs at inference produce out-of-distribution perceptual embeddings that cause behavioral divergence without the model explicitly learning adversarial outputs.

Key Contributions¶

First formalization of backdoor attack surfaces in end-to-end VLA models under the TaaS threat model.
Objective-decoupled two-stage optimization that separates trigger encoding (perception module) from task learning (backbone + action head), enabling stealthy and architecture-agnostic injection.
Demonstration that naive baseline approaches (BadNet-style data poisoning and UADA model poisoning) fail entirely on VLA benchmarks (ASR = 0%), while BadVLA consistently succeeds.
Empirical evidence that the injected backdoor survives JPEG compression, Gaussian noise, and downstream re-fine-tuning.

Results¶

OpenVLA on LIBERO (4 task suites, 3 trigger types – block, mug, stick):
Average ASR: 98.3% (block), 97.8% (mug), 96.1% (stick) with clean-task SR degradation of ≤3.4% from baseline (~96.7%).
Baseline data-poisoning and model-poisoning methods: SR (w/o) = 0.0 and ASR = 0.0 on all tasks.
SpatialVLA on SimplerEnv (3 google_robot tasks): ASR up to 100% (pick_object, move_near) with 0% SR (w/) while SR (w/o) matches the 70–80% baseline.
Compression robustness (q = 20%): Average ASR 97.7%, SR (w/o) 95.2%.
Gaussian noise robustness (ε = 0.08): Average ASR 94.1%, SR (w/o) 91.3%.
Post-re-fine-tuning (cross-task): ASR remains 90.6–97.9% after adaptation; clean SR recovers to >87%.
Ablation: Removing trigger separation loss (L2) collapses ASR to ~1.2%; removing reference alignment (L1) collapses clean SR to 38.3% (Libero_10); removing Stage II collapses both metrics.
Feature space: Cosine similarity between clean and triggered embeddings drops from 0.98 (pre-attack) to 0.21 after Stage I.

Limitations¶

Only untargeted backdoor attacks are studied; targeted attacks (forcing a specific malicious action) are left to future work.
Physical-world deployment of triggers is not validated; experiments are simulation-only (LIBERO, SimplerEnv).
The white-box assumption (full access to architecture and pretrained weights) may not hold in all TaaS scenarios.
Instruction-level (language modality) backdoors are infeasible in current VLAs because OpenVLA-class models show weak instruction-to-action grounding (output actions are near-identical regardless of instruction perturbation).
Proposed mitigations (perceptual-feature monitoring, model distillation) are acknowledged as partial and not formally evaluated.

Relevance to Vision-Language Models¶

BadVLA directly targets the VLM backbone of VLA systems (e.g., LLaMA2-based OpenVLA), showing that the same open-weight ecosystem that enables rapid downstream fine-tuning also creates a persistent backdoor surface. The attack exploits the tightly coupled, opaque multimodal representation learned by vision-language pretraining: by corrupting only the visual encoder's latent space, it hijacks downstream action generation without touching the language or action heads. For researchers tracking VLMs, this work highlights that security threat models must be extended beyond classification/generation tasks to embodied, sequential decision-making settings, and that the standard VLM fine-tuning pipeline (freeze encoder, adapt head) can itself become a vulnerability vector.