Hybrid Training for Vision-Language-Action Models¶

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; hybrid VLA training combining CoT with diffusion-based actions

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Hybrid Training (HyT) is a framework that trains Vision-Language-Action (VLA) models on both chain-of-thought (CoT) reasoning traces and direct action prediction via a modality variable, enabling the model to internalize CoT knowledge without generating thoughts at inference time. This decouples the performance benefits of embodied CoT from its inference-time cost, matching ECoT accuracy at standard VLA speed (~3 Hz vs. ECoT's ~1 Hz). The core insight is that CoT's value lies primarily in shaping learned representations, not in runtime thought generation.

Problem¶

Embodied Chain-of-Thought (ECoT) methods improve VLA task performance by generating intermediate reasoning before actions, but this 3–4× increases inference latency—critical in real-time manipulation where actions must be produced at high frequency. Standard VLAs lack the representational richness from reasoning supervision. The gap: can a single model capture CoT's representational benefits without paying its inference cost?

Method¶

HyT introduces a modality variable \(m \in \{m_a, m_\tau, m_f\}\) that conditions the VLA's output distribution. The unified objective marginalizes over thoughts \(\tau\) and modality values:

\[p(a_t|x_t, l) = \sum_i \sum_j p_\theta(a_t, \tau^i | x_t, l, m_j)\, p(m_j)\]

Three conditional distributions are trained jointly: - Act (\(m_a = \texttt{<act>}\)): direct action prediction, \(p_\theta(\tau=\emptyset|m_a)=1\) - Think (\(m_\tau = \texttt{<think>}\)): ECoT-style thought-then-action generation - Follow (\(m_f\)): conditions actions on externally provided thoughts/instructions

The training loss is \(\mathcal{L}_\text{hyt}(\theta) = w_\alpha \mathcal{L}_\text{act} + w_\tau \mathcal{L}_\text{think} + w_f \mathcal{L}_\text{follow}\), with coefficients \(\{w_a{:}0.25,\, w_\tau{:}0.5,\, w_f{:}0.25\}\) used as Monte Carlo sampling probabilities (not loss weights directly) to maintain batch diversity. The model is PaliGemma-2 (3B), fully fine-tuned from the VLM. At inference, \(m_a = \texttt{<act>}\) is set by default, producing actions at the same speed as a standard VLA.

Key Contributions¶

Hybrid Training objective: a single model learns act/think/follow distributions via a modality token and Monte Carlo sampling of mixed batches
Inference-time flexibility: same checkpoint can generate actions directly, produce interpretable thoughts, or follow externally provided language instructions
Empirical validation of the representation hypothesis: CoT's primary benefit is representational, not generative—fast "act" mode matches "think" mode performance after HyT training
LIBERO SOTA: HyT achieves 93.7% average across four LIBERO task suites, outperforming all listed baselines including \(\pi_0\)-FAST (85.5%) and VLA-OFT (92.1%)
Maintains ~3 Hz inference (same as standard VLA); ECoT runs at ~1 Hz, HiRobot at ~0.75 Hz

Results¶

ClevrSkills (9 tasks, 300–3000 demos): HyT outperforms VLA, ECoT, and HiRobot at all dataset sizes; gains are largest for complex, long-horizon tasks
Inference frequency (A100, 4 parallel models): HyT ≈ 3 Hz; ECoT ≈ 1 Hz (3× slower); HiRobot ≈ 0.75 Hz (4× slower)
LIBERO (100 eval episodes per suite):
HyT: Spatial 94.0, Object 97.2, Goal 96.2, Long 89.4, Avg 93.7
VLA-OFT: Avg 92.1; \(\pi_0\)-FAST: Avg 85.5; CoT-VLA: Avg 81.1; OpenVLA: Avg 76.5
Instruction following (ClevrSkills): oracle-provided thoughts in "follow" mode improve over self-generated thoughts; "act" and "think" modes yield similar success rates without oracle input

Limitations¶

Thoughts used in training are simple (current subtask + coarse directional motion); it is unverified whether HyT's inference-time equivalence between "act" and "think" modes holds for tasks requiring complex embodied reasoning
Dynamic switching of the modality token during an episode is not explored
Thought annotations must be available or generated (e.g., via bounding-box labeling from simulation); scalability to unstructured data is not addressed
Hierarchical and oracle-assisted comparisons use oracle thoughts only on "move" subtasks due to policy imprecision on pick/place conditions, limiting the scope of the follow-mode evaluation

Relevance to Vision-Language Models¶

HyT directly addresses how VLMs fine-tuned for action prediction can leverage chain-of-thought supervision without sacrificing real-time usability—a core tension in adapting LLM reasoning techniques to embodied domains. The result that CoT improves representations rather than requiring runtime thought generation aligns with emerging work on reasoning pre-training and distillation in language models, and has implications for how VLM training curricula should be designed. The modality-token mechanism is a lightweight architectural addition to any autoregressive VLM backbone, making findings broadly transferable. For VLM researchers, this is concrete evidence that structured intermediate supervision reshapes internal representations in ways that persist at inference time even when the supervision signal is absent.