Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models

🕒 Published (v1): 2026-07-16 06:59 UTC · Source: Arxiv · link

Why this paper was selected

Action supervision reshapes multimodal representations in VLA; structural insight for builders

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Action QFormer introduces a query-based action-facing interface inserted between a pretrained vision-language backbone and a downstream policy head in VLA models. It addresses a dual effect of action supervision: action-loss gradients are necessary to form action-compatible representations, but when they propagate directly through the inherited multimodal pathway they also destabilize language-side processing and object grounding. On zero-shot sim-to-real navigation, Action QFormer raises closed-loop task success from 18.8% to 56.3% and fixed-instruction action correctness from 22.5% to 75.5%.

Problem

VLA models reuse pretrained VLM backbones for embodied control, but action finetuning applies action-loss gradients that propagate through the same inherited multimodal pathway that must also support language generation and object grounding. This creates a dual effect: action-supervised shaping is necessary to produce action-compatible representations, yet direct exposure of the inherited pathway to action-loss gradients causes broad upstream rewriting that destabilizes instruction generation and visual grounding. Prior work on action interfaces (tokenization, decoding, cross-embodiment transfer) leaves how action supervision reshapes inherited representations largely unexplored.

Method

Action QFormer is a \(K\)-layer query-based action-facing interface placed between the pretrained multimodal backbone and the policy head. It maintains \(M\) learnable query states \(Q^0 \in \mathbb{R}^{M \times d}\). At each layer \(k\):

\[\tilde{Q}^k = \text{SelfAttn}^k\!\left(\text{Concat}(Q^{k-1}, H_t^S)\right)_{\text{query}}, \quad Q^k = \text{CrossAttn}^k(\tilde{Q}^k, H_t^I)\]

Self-attention conditions queries on instruction-side representations \(H_t^S\); cross-attention lets the instruction-conditioned queries extract visual evidence from image-side representations \(H_t^I\). Final query outputs are pooled and concatenated with \(H_t^S\) to form the action-facing representation \(z_t^{AQF}\).

Critically, this changes the backward gradient route: action-loss updates act on the query interface parameters \(\theta_{AQF}\) and learnable query initialization \(Q^0\) before propagating upstream to \(\theta_{\text{path}}\), giving action supervision an intermediate adaptation path that reduces direct rewriting of inherited representations.

Training combines a language generation objective \(\mathcal{L}_{\text{lang}}\) (teacher-forced over GPT-generated navigation instructions) and a diffusion denoising action-prediction objective \(\mathcal{L}_{\text{action}}\), combined as \((1-r)\mathcal{L}_{\text{action}} + r\mathcal{L}_{\text{lang}}\) with \(r\) cosine-annealed from high to low. Stop-gradient diagnostic ablations across four gradient settings (action blocked, language backbone blocked, vision encoder frozen, full update) are used to isolate the dual effect. Evaluation is zero-shot sim-to-real ObjectNav: trained in Habitat simulation, deployed directly in real indoor scenes.

Key Contributions

• Formulates structured representation shaping under action supervision as a mechanism-level design problem in VLA, distinguishing necessary action-compatible adaptation from disruptive upstream rewriting.
• Introduces Action QFormer: instruction-conditioned learnable queries that reorganize inherited image-side and instruction-side representations into action-facing representations before the policy head, while mediating the backward gradient route.
• Demonstrates large zero-shot sim-to-real gains in closed-loop navigation without any domain adaptation finetuning.
• Provides mechanistic analysis via stop-gradient interventions showing that the query interface reduces broad upstream rewriting, strengthens directional distinctions, and stabilizes instruction-to-visual attention.

Results

All numbers are from zero-shot sim-to-real navigation (trained in Habitat, deployed in real scenes); baseline is the direct-fusion VLA:

• Closed-loop task success: 18.8% (direct fusion) → 56.3% (Action QFormer)
• Fixed-instruction action-generation correctness: 22.5% → 75.5%
• Out-of-distribution instruction generation: nearly eliminated by Action QFormer
• Representation-level analysis: Action QFormer strengthens directional distinctions in action-facing representations while reducing upstream gradient rewriting of inherited backbone features.

(Full Table II results were truncated in the provided text.)

Limitations

• Evaluated only on goal-conditioned indoor navigation (ObjectNav); generalization to manipulation or other VLA domains is unverified.
• Uses a single-frame observation model; temporal context across steps is not encoded in the backbone, only in the closed-loop execution loop.
• Training data is entirely simulation-derived (Habitat); real-to-sim appearance gap is an acknowledged challenge but not systematically characterized.
• The number of query states \(M\) and layers \(K\) are architectural hyperparameters whose sensitivity is not fully characterized in the visible text.
• Mechanistic claims about gradient routing rely on stop-gradient diagnostics, which are informative but do not constitute causal proof.

Relevance to Vision-Language Models

This paper directly concerns the tension between preserving pretrained VLM representations and adapting them for a new modality (action), a problem increasingly central as VLMs are extended to embodied settings. The Action QFormer architecture is an actionable extension of query-based VLM interfaces (BLIP-2, InstructBLIP, Flamingo) from multimodal alignment to action-generation contexts, showing that the gradient route — not just the forward architecture — is a first-class design variable. The finding that direct action finetuning destabilizes language-side processing of the backbone is a concrete mechanistic insight for anyone studying catastrophic forgetting or representation interference in VLM fine-tuning. More broadly, the structured-shaping framing reframes VLA action interfaces as representation mediators, suggesting that query-based bottlenecks are a principled strategy for modality extension of pretrained VLMs beyond vision-language tasks.