Skip to content

Reasoning as a Double-Edged Sword: Architecture and Cross-Stage Robustness in Vision-Language-Action Models

🕒 Published (v1): 2026-07-20 10:20 UTC · Source: Arxiv · link

Why this paper was selected

Empirical robustness audit of reasoning in VLAs; honest baselines; safety-critical insight

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper tests whether adding explicit reasoning to Vision-Language-Action (VLA) models improves robustness to perturbations. Counterintuitively, the latent-iterative model (RD-VLA) is the least robust, collapsing to 0% task success under white-box PGD-10 while a text-CoT model and a no-reasoning model survive. This fragility is shown to be structural (K-invariant) rather than depth-cumulative, and reasoning outputs fail as runtime safety monitors under adaptive attack.

Problem

Existing VLA robustness benchmarks treat models as black boxes and never isolate the reasoning stage, leaving untested whether reasoning attenuates, relays, or amplifies adversarial perturbations, and whether exposed reasoning can serve as a reliable runtime safety signal.

Method

Three LIBERO-fine-tuned VLA models spanning the reasoning spectrum are evaluated: - OpenVLA-OFT (~7B, no reasoning, Prismatic+MLPResNet action head) - DeepThinkVLA (DT) (~3B, text chain-of-thought, PaliGemma backbone, discrete action tokens) - RD-VLA (0.5B, latent iterative recurrence \(f_r = g^{\circ K}\) with \(K{=}12\), Prismatic backbone, linear action projection)

A cross-stage attack matrix injects perturbations at each pipeline stage independently: vision (FGSM, PGD-10 at \(\ell_\infty\) \(\varepsilon \in \{4,8,16\}/255\); Gaussian \(\sigma \in \{0.01\text{–}0.2\}\)), reasoning (entity-swap CoT corruption, DT only), and action (Gaussian on 7-DOF output). White-box PGD-10 through RD-VLA's \(K{=}12\) recurrence is enabled by activation checkpointing. Evaluation uses \(N{=}12\) matched (suite, seed) cells (4 LIBERO suites × 3 seeds), with Holm–Bonferroni correction and bootstrap CIs.

An inference-time \(K\)-sweep (\(K \in \{4, 8, 12\}\)) tests whether depth drives fragility by measuring the amplification ratio \(\rho(K) = \|\Delta a\|_2 / \varepsilon\).

Runtime monitors are evaluated under Carlini adaptive attack: (1) a plan–action consistency probe scoring CoT–instruction agreement via \(s = 0.5\,S_\text{entity} + 0.2\,S_\text{direction} + 0.15\,S_\text{gripper} + 0.15\,S_\text{parse}\); (2) a stage-fused monitor \(s_t = \alpha(1-\varphi_t) + (1-\alpha)\psi_t\) combining the consistency probe with an action-anomaly Mahalanobis probe, calibrated at a matched 8% clean FPR across all fusion weights \(\alpha\).

Key Contributions

  • Structural fragility, not cumulative: RD-VLA collapses under both Gaussian noise and PGD-10, while DT and OFT do not; \(\rho(12)/\rho(8)=1.005\) against the multiplicative Lipschitz prediction of \(2.0\) (\(\hat{L}_\text{iter}=1.192\)), falsifying the per-iteration compounding hypothesis and localizing the amplifier to the encoder and the recurrence's fixed point.
  • Three-tier PGD-10 ordering: DT (49.8%) \(\gg\) OFT (18.2%) \(\gg\) RD-VLA (0.0%) at \(\varepsilon{=}8/255\), establishing text CoT as more robust than single-pass under white-box attack.
  • Adaptive monitor ceiling: The consistency probe collapses from AUC 0.996 (naive) to 0.493 (stealth entity re-injection) and 0.259 (full adaptive); stage-fused probes never lift defended SR above undefended SR under matched FPR — characterized as a precondition for defense work, not a no-go theorem.

Results

  • Gaussian \(\sigma{=}0.2\): RD-VLA 14.8% SR (\(\Delta = -74.3\,\text{pp}\)); DT 92.7%; OFT 89.0% (DT vs. OFT inconclusive, \(p{=}0.30\); both vs. RD-VLA \(p{<}0.01\), \(d{=}6.5\))
  • PGD-10 \(\varepsilon{=}8/255\): DT 49.8%, OFT 18.2%, RD-VLA 0.0%
  • \(K\)-sweep: \(\rho(K) = 48.507 \cdot 1.0007^K\) (MAFE 0.1%); \(\rho(12)/\rho(8){=}1.005\) vs. multiplicative prediction of \(2.0\)
  • Amplification ratio: \(\rho_\text{RD-VLA}/\rho_\text{OFT} = 8.22\) at \(\sigma{=}0.2\)
  • Consistency probe AUC: 0.996 (naive entity-swap) → 0.493 (stealth) → 0.259 (full adaptive); vision-stage attacks defeat it without adaptation (AUC \(\le 0.692\) for all FGSM levels; 0.463 at Gaussian \(\sigma{=}0.2\))
  • Stage-fused monitor (\(\alpha{=}0.5\)): 42.2% defended SR vs. 49.8% raw SR under PGD-10 (text-only baseline at \(\alpha{=}1\) recovers 48.8%)
  • LIBERO-Plus naturalistic perturbations: RD-VLA gap \(> 30\,\text{pp}\) on 6/7 factors
  • SimplerEnv: PGD-10 exceeds FGSM by mean 17.2 pp across tasks on OpenVLA-7B (replicates LIBERO asymmetry)

Limitations

  • Visual reasoning paradigm (CoT-VLA) is out of scope; only text CoT and latent iterative evaluated.
  • Physical-world transfer of perturbations is explicitly deferred to future work.
  • Stage-level Lipschitz decomposition isolating encoder vs. recurrence contributions remains incomplete.
  • CoT-disabled ablation on DT uses an off-manifold zero-token control rather than an in-distribution skip-CoT comparator.
  • SimplerEnv analysis is single-architecture only (OpenVLA-7B), limiting cross-model replication.
  • FGSM on RD-VLA uses DT→RD-VLA transfer as a white-box lower bound (direct white-box FGSM not run).
  • Adaptive ladder tests only entity-swap corruption; direction- and gripper-state swap evasion deferred.

Relevance to Vision-Language Models

This work directly probes how reasoning architecture in VLMs affects adversarial robustness at inference time, revealing that weight-tied latent recurrence — a design present in iterative refinement and some recurrent LLM variants — produces amplification that is encoded structurally in the encoder–fixed-point coupling rather than accruing with depth. The K-invariance result (\(\rho(12)/\rho(8){\approx}1\)) challenges the standard Lipschitz-multiplicative intuition that motivates capping recurrence depth as a safety measure. For VLM safety researchers, the failure of chain-of-thought consistency probes under adaptive attack quantifies a fundamental limitation: exposing the reasoning trace does not straightforwardly enable reliable anomaly detection when the attacker can steer the trace coherently with the corrupted action. The study thus sets an empirical ceiling that any output-level behavioral monitor for VLMs with text reasoning must first surpass before being claimed as a viable defense.