VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision–Language Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; VITA: test-time adaptation of VLMs as zero-shot goal-conditioned value functions

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VITA adapts frozen contrastive VLMs (CLIP) at test time to serve as zero-shot goal-conditioned value functions for robotic manipulation. A lightweight adaptation module is updated online via a meta-learned self-supervised reconstruction loss, encoding trajectory history implicitly into parameters. VITA generalizes across unseen tasks, environments, and robot embodiments without expert demonstrations, and its reward estimates enable offline RL policies that outperform those trained with dense hand-crafted rewards.

Problem

Frozen pre-trained VLMs used as zero-shot value functions suffer from two distinct failure modes: contrastive VLMs (CLIP) lack temporal context and confuse visually similar states at different task stages; autoregressive VLMs have a monotonicity bias from chronologically ordered pre-training data. Both fail to generalize under distribution shifts without large-scale domain-specific pre-training or expert demonstrations.

Method

VITA builds a three-component estimator: (1) a frozen OpenCLIP ViT-B/32 encoder produces concatenated visual–language representations \(z_t = [\phi_v(o_t); \phi_g(g)] \in \mathbb{R}^{2d}\); (2) a lightweight adaptation module \(f_\text{adapt}\) updated at each inference timestep \(t\) by a single gradient step on a meta-learned self-supervised reconstruction loss: $\(\ell_\text{self}(z_t; \theta_{t-1}, P_K, P_V) = \| f_\text{adapt}(P_K z_t; \theta_{t-1}) - P_V z_t \|^2\)$ $\(\theta_t = \theta_{t-1} - \eta \nabla_\theta \ell_\text{self}\)$ Sequential parameter updates encode trajectory history implicitly — analogous to an RNN hidden state but stored in weights; (3) a frozen two-layer MLP regression head \(h\) that outputs \(V(z_t; g) = h(f_\text{adapt}(P_Q z_t; \theta_t)) \in [0,1]\).

Training uses gradient-based meta-learning (MAML-style): differentiate through test-time update steps to optimize \(P_K, P_V, P_Q, \theta_0\), and \(h\) jointly against MSE on normalized timestep progress labels. To prevent shortcut learning from temporally redundant video frames, a dissimilarity-based sampling strategy selects the \(k\) most visually diverse sliding-window sub-trajectories per batch by maximizing pairwise \(\ell_2\) dissimilarity, approximated greedily via per-window diversity scores.

Key Contributions

• VITA: a test-time training (TTT) framework applied to goal-conditioned value function estimation, requiring no task-specific demonstrations and no large-scale domain pre-training.
• Dissimilarity-based sampling to reduce shortcut learning from temporally correlated frames.
• Zero-shot generalization demonstrated across task, environment, and embodiment shifts on BridgeData V2 real-robot data.
• Zero-shot reward shaping on Meta-World MT10 offline RL benchmark, surpassing policies trained with the simulator's own fuzzy-logic dense rewards.

Results

• VOC (generalization): VITA outperforms CLIP-GRU on 6/10 evaluation splits and dominates GVL (Gemini 1.5 Pro, zero-shot and one-shot) on all 10; e.g., in-distribution (tk pnp): VITA 0.782 vs. GVL-1S 0.252; long-horizon sweep (ms sweep): VITA 0.490 vs. GVL-1S 0.150; embodiment+environment shift (dt rd pnp): VITA 0.695 vs. GVL-1S 0.316.
• BinVOC (expert vs. non-expert discrimination): VITA achieves 1.00 (perfect), matching GVL-0S/1S and exceeding CLIP-GRU (0.80) and CLIP-FT (0.80).
• Offline RL on Meta-World MT10 (IQM): VITA 0.815 [0.785, 0.838] vs. best non-VITA baseline META-WL 0.779, CLIP-FT 0.785; all baselines using the simulator's own dense rewards are surpassed.

Limitations

• Trained exclusively on a single robot embodiment (WidowX 250) in four ToyKitchen configurations; generalization is tested but not trained cross-embodiment.
• Training data (BridgeData V2) covers only pick-and-place; folding, sweeping, stacking are OOD — some VOC scores decline on these (e.g., ft fold: 0.658 vs. CLIP-GRU 0.693).
• Temporal supervision assumes monotonically increasing expert progress, which may not hold for complex or non-linear tasks.
• A single gradient step at test time is efficient but may be insufficient for highly complex temporal dynamics; the sensitivity to hyperparameter choices (learning rate \(\eta\), \(\lambda_\text{self}\), window size) is reported but not fully ablated across embodiment domains.
• Offline RL evaluation uses simulated Meta-World data for policy training, not real-robot execution, limiting direct real-world validation of the reward shaping claim.

Relevance to Vision-Language Models

VITA directly addresses a practical limitation of deploying VLMs as dense reward or progress signals: their frozen representations cannot capture task-specific temporal dynamics. The test-time adaptation mechanism is a lightweight alternative to fine-tuning or in-context learning with large autoregressive VLMs (Gemini 1.5 Pro), achieving superior performance at a fraction of the inference cost. For researchers tracking VLMs, this demonstrates that parameter-efficient adaptation at inference — rather than scale or prompting — can unlock temporal reasoning in CLIP-class models. The work also highlights an underexplored application of VLMs as value function estimators that transfer zero-shot across embodiments, providing a principled bridge between vision-language pre-training and embodied RL reward design.