Skip to content

Verifier-free Test-Time Sampling for Vision-Language-Action Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; verifier-free test-time scaling for VLA precision tasks

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MG-Select is a training-free (or fine-tune-only) test-time scaling framework for autoregressive Vision-Language-Action (VLA) models that selects the best action from N sampled candidates using KL divergence between the model's full-condition distribution and a condition-masked reference distribution as a confidence proxy. It requires no external verifier and generalizes to unseen conditions. Experiments on RoboCasa, SIMPLER-WidowX, LIBERO, and a real Franka robot show consistent precision improvements over strong baselines.

Problem

Autoregressive VLAs rely on greedy (single-inference) decoding, which bottlenecks precision on fine-grained manipulation tasks (e.g., grasping, object placement). Prior test-time scaling solutions use externally trained value-function verifiers, which incur heavy RL training overhead and fail to generalize to novel task prompts, objects, or environments not seen during verifier training. Simple likelihood-based Best-of-N selection also fails because VLAs fine-tuned on expert demonstrations collapse their token distributions, causing repeated samples to converge to the same action.

Method

MG-Select operates as a two-stage test-time procedure on any autoregressive VLA \(\pi_\theta\):

  1. Parallel stochastic sampling. At timestep \(t\), \(N\) candidate action sequences \(\tilde{a}^{(n)}\) are sampled with temperature \(\tau > 0\) from \(\pi_\theta(a \mid o_t, q_t, I)\).

  2. Condition-masking distributional confidence. For each candidate, a token-level confidence score is computed as the KL divergence from a condition-masked reference distribution to the full-condition predicted distribution: $\(C_{\tilde{a}} = \sum_{i \in \mathcal{I}} \mathrm{KL}\!\left(\pi_\theta(\cdot \mid o_t, q_t^{(m)}, I^{(m)}, \tilde{a}_{<i}) \;\Big\|\; \pi_\theta(\cdot \mid o_t, q_t, I, \tilde{a}_{<i})\right)\)$ where \((q_t^{(m)}, I^{(m)})\) masks proprioceptive state, language instruction, or both. The reference distribution is produced by the same VLA with masked inputs, requiring no additional model. The candidate with the highest \(C_{\tilde{a}}\) is selected (\(a^* = \arg\max C_{\tilde{a}^{(n)}}\)).

  3. Joint training strategy (MG-Select*). To produce better-calibrated masked distributions, the VLA is fine-tuned on an augmented dataset containing all four masking variants \(\mathcal{M} = \{(q_t, I), (q_t, \emptyset), (\emptyset, I), (\emptyset, \emptyset)\}\) via a joint imitation learning objective: $\(\mathcal{L}_{\text{Joint-IL}}(\theta; \mathcal{D}) = -\mathbb{E}\left[\mathbb{E}_{(q_t^{(m)}, I^{(m)}) \in \mathcal{M}} \log \pi_\theta(a_t \mid o_t, q_t^{(m)}, I^{(m)})\right]\)$ The masking variant used at test time (text, state, or both) is selected based on task structure.

Key Contributions

  • A verifier-free Best-of-N test-time scaling framework for autoregressive VLAs using condition-masking KL divergence as a confidence metric.
  • The condition-masking reference distribution design: masked inputs produce task-aligned but uncertain distributions that are more informative than uniform or likelihood-based references.
  • A joint training strategy with condition dropout that improves the quality of the masked reference without sacrificing standard task performance.
  • Empirical validation across three simulation benchmarks (RoboCasa, SIMPLER-WidowX, LIBERO) and real-world Franka robot experiments, including out-of-distribution generalization.

Results

  • RoboCasa (30 demos, pick-and-place): Ï€0-FAST base 5.3% → MG-Select* 14.2% (168% relative gain); all-24-task: 30.9% → 34.6%.
  • RoboCasa (100 demos, pick-and-place): 17.0% → 31.0% (+82% relative); all tasks: 40.2% → 48.1%.
  • RoboCasa (300 demos): 43.2% → 46.9% (pick-and-place), 61.2% → 62.9% (all).
  • SIMPLER-WidowX (4 pick-and-place tasks): Ï€0-FAST 46.9% → MG-Select* 50.3% average; outperforms RT-1-X (1.1%), Octo (16.0%), RoboVLM (31.3%), SpatialVLA (42.7%).
  • Real-world ID (Franka, 4 pick-and-place tasks): 37.5% → 47.9% (+28% relative).
  • Real-world OOD (unseen objects): 53.1% → 71.9% (+35% relative).
  • Ablation (Table 5a): MG-Select (48.1%) > Likelihood Best-of-N (46.8%) > Uniform-KL Best-of-N (46.5%) > Greedy (42.7%) on RoboCasa 100-demo all tasks.
  • Performance scales with \(N\) up to 64 candidates; gains are marginal beyond \(N=4\).
  • Optimal token aggregation uses "First 5" tokens of action sequence (48.1% vs. full-sum 44.5% on RoboCasa pick-and-place).

Limitations

  • Optimal masking variant (text, state, or both) is task-dependent and requires environment-level selection heuristics rather than automatic adaptation.
  • Inference latency scales linearly with \(N\) (generating \(N\) candidates + \(4\times\) masked forward passes per candidate).
  • Evaluation is confined to pick-and-place tasks; performance on more diverse manipulation primitives (e.g., articulated object manipulation, tool use) is not demonstrated.
  • Real-world experiments use only one robot platform (Franka Research 3) with limited task diversity.
  • Joint training requires access to the fine-tuning data and adds training cost (though less than RL-based verifier training).

Relevance to Vision-Language Models

MG-Select directly extends test-time compute scaling—a technique that has produced large gains in text-domain LLMs—to VLA models, which fuse vision, language, and action modalities. The core insight (using self-uncertainty from masked conditioning as a confidence signal) is analogous to self-consistency and self-certainty methods in LLM reasoning, but adapted to continuous robotic action spaces where outcome verifiers are costly or absent. For VLM researchers, this work demonstrates that the internal representations of multimodal models already encode meaningful uncertainty signals accessible without auxiliary networks. The joint training strategy (condition dropout for masked distribution awareness) is directly compatible with existing instruction-following VLM pre-training pipelines and could generalize to other grounded VLM tasks where precise output selection matters.