QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; channel-aware quantization enables VLA deployment on resource-constrained robots

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

QVLA is the first action-centric quantization framework for Vision-Language-Action (VLA) models, arguing that LLM-derived methods fail because they preserve feature fidelity rather than action fidelity. It assigns per-channel bit-widths from \(\{0, 2, 4, 8, 16\}\) guided by direct action-space sensitivity measurements. On OpenVLA-OFT in LIBERO, it retains 98.9% of full-precision performance using only 29.2% of the original VRAM with a 1.49× speedup.

Problem

Existing quantization methods (SmoothQuant, AWQ, OmniQuant) were developed for LLMs and MLLMs, targeting text perplexity and feature reconstruction. Applied naively to VLA models, they fail to account for: (1) the fact that action outputs directly interface with the physical world, so even imperceptible quantization errors compound autoregressively into catastrophic failures over long-horizon tasks; and (2) heterogeneous sensitivity across both modules (projector and action head are far more sensitive than the vision encoder) and individual channels within the same layer.

Method

QVLA operates in two stages:

Stage 1 — Action-space sensitivity estimation. For each channel \((l, c)\) quantized to bit-width \(b\), the single-step sensitivity is: $\(s_{l,c}^{(b)} = \mathbb{E}_{x \sim \mathcal{D}} \left\| \tilde{A}_{l,c}^{(b)}(V, l) - A^*(V, l) \right\|_2^2\)$ A cumulative variant \(S_{l,c}^{(b)} = \mathbb{E}\left[\sum_{t=1}^T \| \tilde{A}_{l,c}^{(b)}(V_t, l) - A^*(V_t, l) \|_2 \right]\) captures long-horizon error accumulation. Exhaustive evaluation is made tractable via a Taylor-series first-order proxy: \(\|\Delta A\| \approx \|J_{A,X_{l,c}}\| \cdot \|\Delta X_{l,c}\|\), where \(J_{A,X_{l,c}}\) is the Jacobian of the action w.r.t. channel output. This proxy ranks all channels cheaply; full forward passes are only run for high-priority channels.

Stage 2 — Greedy demotion bit allocation. All channels start at 16-bit. The algorithm iterates through demotion stages \(16 \to 8 \to 4 \to 2 \to 0\), sorting channels at each stage by the sensitivity-to-bit ratio: $\(\rho_{l,c} = \frac{s_{l,c}^{(b_\text{lo})} - s_{l,c}^{(b_\text{hi})}}{b_\text{hi} - b_\text{lo}}\)$ and demoting the least sensitive (lowest \(\rho_{l,c}\)) first until the average bit budget \(\bar{B}\) is met. Assigning \(0\)-bit unifies pruning into the same framework. Activations use uniform bit-width for hardware-friendly, branch-free execution; weights use per-output-channel integer quantization. Complexity is \(O(C \log C)\).

Key Contributions

• First systematic empirical analysis of VLA-specific quantization sensitivity, revealing module-level and channel-level heterogeneity.
• QVLA: a channel-wise, action-space-anchored mixed-precision framework that cohesively unifies weight quantization and structural pruning.
• Efficient sensitivity estimation via Taylor-series proxy combined with selective precise evaluation.
• Greedy demotion algorithm with dual-threshold and L0-style regularization to prevent over-pruning.

Results

All results on LIBERO benchmark (four task suites: Spatial, Object, Goal, Long); baselines are SmoothQuant and OmniQuant.

• OpenVLA-OFT W4A4: QVLA 96.0% avg (\(\Delta\) −1.1%) vs. SmoothQuant 73.4% (−23.7%) and OmniQuant 93.9% (−3.2%); VRAM 4.5 GB vs. 15.4 GB FP; speedup 1.49×.
• OpenVLA W4A4: QVLA 76.0% avg (−0.5%) vs. SmoothQuant 63.2% (−13.3%) and OmniQuant 73.3% (−3.2%); VRAM 4.3 GB vs. 15.2 GB FP; speedup 1.47×.
• OpenVLA-OFT W8A8: QVLA 96.4% (−0.7%) vs. SmoothQuant 96.0% (−1.1%) and OmniQuant 94.3% (−2.8%); VRAM 7.2 GB; speedup 1.36×.
• QVLA achieves a 22.6% performance improvement over SmoothQuant at W4A4 on OpenVLA-OFT.

Limitations

• Activations are restricted to uniform bit-width across the model to avoid runtime kernel branching; only weights receive per-channel mixed precision.
• The sensitivity estimation, even with the Taylor proxy, still requires calibration-set forward passes; cost for very large VLA models is not fully characterized.
• Evaluation is confined to the LIBERO simulation benchmark with two backbone architectures (OpenVLA, OpenVLA-OFT); real-robot transfer is not demonstrated in the available text.
• The greedy demotion algorithm is a heuristic for an NP-hard problem and may not find globally optimal bit assignments.

Relevance to Vision-Language Models

VLA models are direct extensions of VLMs, inheriting their vision encoder + projector + LLM backbone architecture, but with an additional action decoder that creates a fundamentally different sensitivity profile. This paper is directly relevant to VLM practitioners because it shows that the projector — the cross-modal bridge common to virtually all VLMs (e.g., LLaVA-style architectures) — is among the most quantization-sensitive components, a finding that may inform compression strategies for VLMs even without action heads. The action-space-anchored sensitivity metric establishes a general principle: quantization objectives should be aligned with the downstream output distribution rather than internal feature reconstruction, which is a transferable insight for any VLM task where perplexity is a poor proxy for downstream metric (e.g., VQA accuracy, grounding IoU). It also benchmarks the failure modes of SmoothQuant and OmniQuant on multimodal models, providing concrete evidence of their limitations in cross-modal settings.