FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation¶

🕒 Published (v1): 2026-06-18 18:54 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; systematic few-shot stress test of VLA adaptation with future-oriented conditioning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

FOCA augments pretrained Vision-Language-Action (VLA) models with two complementary future-conditioning objectives—explicit latent prediction of object-centric interaction regions and implicit contrastive alignment to future goal observations—to enable data-efficient adaptation from very few robot demonstrations. By operating entirely in embedding space and supporting action-free co-training on synthetic video rollouts, FOCA achieves state-of-the-art few-shot imitation learning without pixel-level reconstruction. Systematic stress tests reveal that current VLA models degrade sharply below ~40% demonstration budget, a gap FOCA substantially closes.

Problem¶

State-of-the-art VLA models (π0, GR00T-N1.5, EO-1) suffer significant performance collapse under few-shot imitation learning (10–30 demonstrations), yet this regime has not been systematically benchmarked. Standard behavioral cloning treats demonstrations as independent $(o_t, a_t)$ pairs, discarding the rich temporal supervision embedded in future frames—supervision that could densify the learning signal per trajectory. Existing auxiliary objectives (pixel-level future prediction, PEFT with LoRA/DoRA) do not adequately exploit long-horizon structure and remain data-hungry.

Method¶

FOCA is a plug-and-play adaptation framework that injects two auxiliary token sets into the frozen pretrained VLM $F_\theta$ alongside the original vision and language tokens:

Explicit future latent prediction ($r^{exp}_t$): A grounding model $G(\cdot)$ localizes task-relevant objects and the gripper in a future frame $I^i_{t+}$, producing a minimal enclosing interaction bounding box $B^i_{t+}$. Learnable explicit tokens $r^{exp}_t \in \mathbb{R}^{n_e \times d}$ are appended to the VLM input; their output is mapped by a lightweight predictor $P^{exp}_\omega$ (optionally a small Transformer) to produce $\hat{y}^i_{t+}$. The target is patch features from the frozen vision encoder restricted to $B^i_{t+}$: $$L_{exp} = \mathbb{E}_{t,t+,i}\left\|\hat{y}^i_{t+} - \tilde{y}^i_{t+}\right\|^2_2$$

Implicit future alignment ($r^{imp}_t$): Implicit tokens are initialized from pooled vision encoder features, projected via $W^{imp}$. Their VLM output is mapped to a contrastive embedding $\hat{z}_t$, aligned to future interaction embeddings $y^{t+}$ sampled from the same episode using an InfoNCE loss with negatives drawn from task-mismatched episodes in the minibatch: $$L_{imp} = -\mathbb{E}_t\log\frac{\exp(s(\hat{z}_t, y^{t+})/\lambda)}{\sum_{y \in \{y^{t+}\} \cup \mathcal{N}^{\neg task}}\exp(s(\hat{z}_t, y)/\lambda)}$$ The future offset $k = t^+ - t$ is sampled from a truncated geometric distribution with discount $\gamma$, which connects $L_{imp}$ theoretically to goal-conditioned discounted occupancy $\rho^\mu(g | x_t)$ (Theorem 4.1), and $L_{exp}$ to its conditional mean (Proposition 4.2).

A token-isolation mask blocks cross-attention between $r^{exp}_t$ and $r^{imp}_t$ while permitting both to attend to shared vision/language tokens, enforcing functional disentanglement.

The total training objective is $L_{total} = L_{fm} + L_{exp} + L_{imp}$, where $L_{fm}$ is the flow-matching diffusion loss. For action-free synthetic videos (e.g., from DreamGen), only $L_{exp}$ and $L_{imp}$ are optimized, enabling a two-stage curriculum: (i) fine-tune the world model on few-shot demos, generate synthetic rollouts, train with $L_{imp}$; (ii) co-train with real demonstrations using $L_{total}$.

Key Contributions¶

Systematic few-shot stress test: Evaluates π0, GR00T-N1.5, EO-1, SmolVLA at 10%, 40%, and 100% demonstration budgets, quantifying sharp degradation at ≤10%.
Dual future-conditioning objectives: Explicit latent-space interaction prediction + implicit InfoNCE-style goal alignment, operating without pixel-level reconstruction.
Theoretical grounding: Proves that geometric future sampling makes $L_{imp}$ an estimator of goal-conditioned discounted occupancy (log ratio), and $L_{exp}$ an estimator of the occupancy mean.
Action-free co-training: $L_{imp}$ applies directly to synthetic video rollouts without pseudo-actions or inverse dynamics, broadening the usable supervision corpus.
Broad empirical coverage: Validated across 9 simulated benchmarks, a simulated humanoid, and 3 real-robot setups (Aloha arm).

Results¶

LIBERO (few-shot): FOCA (π0-based) achieves 85.3% average at 10% demonstrations vs. π0: 77.6%, GR00T-N1.5: 78.2%, EO-1: 82.2%; 94.0% at 40% vs. π0: 89.9%, GR00T-N1.5: 91.4%.
LIBERO (full data): FOCA 96.6% average vs. π0: 94.6%, GR00T-N1.5: 94.6%, EO-1: 94.1%; strongest across all four LIBERO suites (10, Goal, Object, Spatial).
PEFT comparison (LIBERO, 10%): FOCA 85.3% vs. ControlVLA 78.4%, LoRA 78.2%, DoRA 78.6%.
RoboCasa: 7–12% absolute improvement over baselines (cited in abstract; figure shows ~7.2% and ~10.4% gains over π0 and GR00T-N1.5 on specific tasks).
Real robots: Up to 26% absolute gains on Aloha arm tasks relative to base VLA baselines.
Ablations: Object-centric latent prediction (8 tokens) outperforms full-image pixel prediction and full-image latent prediction; implicit-only achieves 83.6% at 40% vs. explicit-only 93.3%, confirming complementarity.

Limitations¶

Requires a pretrained off-the-shelf grounding model $G(\cdot)$ to construct interaction bounding boxes $B^i_{t+}$; failure modes of the grounding model propagate to $L_{exp}$.
Explicit future prediction requires future frames from the same episode, which are only available during training—not a limitation at inference, but constrains the usable training corpus to episodic demonstrations.
FOCA is evaluated on diffusion-based VLAs (π0, GR00T-N1.5) only; applicability to autoregressive VLA backbones (RT-2, OpenVLA, Gemini) is not demonstrated.
Action-free co-training depends on a video world model (DreamGen) that itself must be fine-tuned on available demonstrations, adding a pipeline stage with its own data and compute cost.
Grounding and future sampling introduce hyperparameters ($\gamma$, $\lambda$, $n_e$, $N_p$) whose sensitivity is evaluated only partially in the ablations shown.

Relevance to Vision-Language Models¶

FOCA directly advances how VLMs are adapted for embodied control: rather than treating the VLM as a frozen feature extractor, it injects future-oriented auxiliary objectives into the VLM's representation during fine-tuning, shaping what the model encodes about scene dynamics and task progress. The implicit alignment objective is a form of contrastive self-supervised learning in the VLM's latent space, closely related to I-JEPA/JEPA-style representation learning but applied to cross-time goal prediction. The theoretical connection to goal-conditioned discounted occupancy bridges VLM-based imitation learning and contrastive RL, suggesting a principled route to grounding VLM representations in long-horizon task value. For researchers tracking VLMs, this work demonstrates that multimodal pretraining alone is insufficient for few-shot robot adaptation—structured future supervision in latent space is necessary—which has implications for both VLA design and VLM fine-tuning methodology more broadly.