Skip to content

Think Less, Act Early: Reinforced Latent Reasoning with Early Exit in Vision-Language-Action Models

🕒 Published (v1): 2026-06-13 04:16 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; reinforced latent reasoning with early exit cuts VLA compute and error propagation

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AVA-VLA replaces explicit Chain-of-Thought reasoning in Vision-Language-Action models with latent reasoning modeled as a POMDP, optimized via RL-based denoising and terminated adaptively by a learned exit gate. The result is a 6× inference speedup over explicit CoT VLA methods with a 98.3% average success rate on LIBERO, outperforming full-reasoning baselines on long-horizon tasks.

Problem

Explicit CoT reasoning in VLA models imposes two compounding costs: (1) high per-step inference latency from token-by-token decoding, which conflicts with real-time robotics requirements, and (2) error propagation where mistakes in early reasoning steps amplify across multi-step sequences, degrading long-horizon task stability.

Method

AVA-VLA models the reasoning process as a sequence of continuous latent states \(z_t \in \mathcal{Z}\) rather than discrete tokens. The framework has three coupled components:

Latent Reasoning as POMDP. The tuple \(\mathcal{M} = (\mathcal{Z}, \mathcal{O}, \mathcal{U}, P, R, \gamma)\) formalizes latent evolution: a reasoning policy \(\pi_\phi(u_t | z_t, o_t) = \mathcal{N}(\mu_\phi, \text{diag}(\sigma_\phi^2))\) emits continuous update actions \(u_t \in \mathbb{R}^{64}\), which drive an incremental state transition: $\(z_{t+1} = z_t + \alpha(u_t) \odot \text{Transformer}_\theta(z_t, \tilde{o}_t)\)$

RL-based Denoising. Without text supervision the latent trajectory is susceptible to drift. A composite reward penalizes task failure, entropy in the update distribution, and large consecutive state changes: $\(r_t = r_\text{task}(a_t) - \lambda_1 H[\pi_\phi(\cdot|z_t, o_t)] - \lambda_2 \|z_{t+1} - z_t\|^2\)$ PPO with GAE optimizes \(\pi_\phi\); a critic \(V^\pi(z_t)\) provides a baseline for variance reduction.

Early-Exit Gate. A lightweight network \(g_\omega(z_t) = e_t \in \mathbb{R}\) measures state confidence; reasoning halts when \(e_t > \tau\) (calibrated to \(\tau = 0.55\)), after which the action policy \(\pi_\psi(a_t | z_t, o_t)\) executes immediately.

Key Contributions

  • AVA-VLA framework: latent-variable formulation of VLA reasoning that bypasses token decoding overhead entirely.
  • RL-based latent denoising via composite reward (task success + entropy regularization + smoothness regularization), optimized with PPO+GAE.
  • Adaptive early-exit strategy that dynamically adjusts reasoning depth per time step based on a learned confidence gate calibrated post-training.
  • Empirical validation on LIBERO and CALVIN ABC→D, covering spatial, object, goal, and long-horizon task suites.

Results

  • LIBERO (single policy across all suites): 98.3% average success rate — highest among all compared methods; Object SR 99.4%, Goal SR 97.8%, Long SR 98.1%.
  • LIBERO (per-suite policy): Spatial SR 99.6%, Object SR 99.7%.
  • CALVIN ABC→D: 84.0% success on 5-step chains; average task length 4.77 — exceeds FLOWER (77.8% / —) and VLA-Adapter (76.5% / —).
  • Inference speedup: 6× over explicit CoT VLA methods.
  • Training requires ~1.2M environment steps; 18.6 hours on 8× A100 80 GB GPUs.

Limitations

  • Latent states are uninterpretable; failure modes cannot be diagnosed from intermediate reasoning, unlike CoT.
  • RL training is compute-intensive (~18.6 hours on 8×A100), limiting accessibility and iteration speed.
  • Evaluation is confined to simulation (MuJoCo/LIBERO, CALVIN); real-robot transfer is not demonstrated.
  • The exit threshold \(\tau\) requires post-training calibration; sensitivity is reported (Table 5) but generalization across task distributions is unclear.
  • LIBERO+ perturbation results are referenced but not fully detailed in the provided text.

Relevance to Vision-Language Models

AVA-VLA sits at the intersection of VLMs and robotic action, directly tackling a central inefficiency of language-model-based reasoning when deployed in low-latency embodied settings. It proposes an alternative to the dominant CoT paradigm — latent reasoning optimized by RL — which is directly relevant to any VLM researcher studying implicit reasoning, adaptive computation, or the "language bottleneck" in multimodal systems (cf. Coconut, Quiet-STaR). The RL-denoising formulation and early-exit mechanism are transferable ideas for VLM inference efficiency beyond robotics. The paper also provides a concrete benchmark comparison against CoT-VLA, SpatialVLA, and FLOWER, making it a useful reference point for the VLM-for-action literature.