Skip to content

Robust Fine-tuning of Vision-Language-Action Robot Policies via Parameter Merging

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Parameter merging preserves generalist VLA skills during task-specific fine-tuning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RETAIN addresses catastrophic forgetting and OOD overfitting when fine-tuning generalist Vision-Language-Action (VLA) robot policies on small demonstration datasets (~50–100 demos). The method is weight-space linear interpolation between the pretrained and fine-tuned checkpoints: \(\tilde{\theta} = (1-\alpha)\theta_\text{pre} + \alpha\theta_\text{ft}\). This single operation simultaneously improves OOD generalization on the new task and preserves the original generalist capabilities.

Problem

Standard fine-tuning of generalist VLA policies (e.g., \(\pi_0\)) on small target-task datasets causes two failure modes: (1) catastrophic forgetting of pretraining capabilities on other tasks, and (2) overfitting to the exact demonstration conditions—the fine-tuned policy fails on OOD variants (new object instances, lighting, viewpoints) even when the base policy had broad generalization. Existing remedies (LoRA, Freeze-FT, Co-FT, data mixing) do not adequately solve OOD generalization in the \(\leq\)100-demo regime.

Method

RETAIN linearly interpolates pretrained weights \(\theta_\text{pre}\) and fine-tuned weights \(\theta_\text{ft}\) with a scalar \(\alpha \in [0,1]\): $\(\tilde{\theta} = (1 - \alpha)\,\theta_\text{pre} + \alpha\,\theta_\text{ft}\)$ Three extensions are introduced: - Co-finetuning (RETAIN-co-FT): fine-tune \(\theta_\text{ft}\) on a mixture of pretraining data \(D_\text{pre}\) and target data \(D_\eta\) before merging. - Modality-specific merging: separate \(\alpha_v, \alpha_l, \alpha_a\) per component (vision encoder, LLM backbone, action expert); empirically merging only the LLM backbone often suffices. - Continual merging: sequential skill acquisition by iteratively merging each new task's fine-tuned weights into the running merged checkpoint: \(\tilde{\theta}_n = (1-\alpha)\tilde{\theta}_{n-1} + \alpha\,\theta_\text{ft,n}\).

The base policies are \(\pi_0\)-FAST-DROID (autoregressive transformer) for real-world experiments and \(\pi_0\) (flow-based action expert) for simulation. Fine-tuning uses behavioral cloning (BC) with the negative log-likelihood objective. \(\alpha\) is tuned over \(\{0.25, 0.5, 0.75\}\) on one held-out OOD validation scene.

Key Contributions

  • First systematic study of weight-space parameter merging for generalist VLA robot policies.
  • RETAIN-task-FT and RETAIN-co-FT achieve state-of-the-art robust fine-tuning with no additional training or inference cost beyond the standard fine-tuning pipeline.
  • Empirical analysis of which model components (LLM backbone vs. vision encoder vs. action expert) benefit most from merging.
  • Demonstration of continual skill accumulation via sequential merging without catastrophic forgetting.
  • Scaling result: RETAIN's effectiveness increases with pretraining data size.

Results

  • DROID real-world (whiteboard, plates): Baselines achieve 30–50% success on OOD (test) scenes versus 70–80% on ID scenes. RETAIN-co-FT achieves >60% on plates and ~80% on whiteboard OOD—matching their own ID performance, ~40% higher OOD success rate than the best baseline on average.
  • LIBERO simulation (3 tasks: pot-on-stove, mugs-on-plates, items-into-basket): Trends mirror DROID; RETAIN outperforms all baselines on OOD and generalist evaluations while remaining competitive on ID.
  • Generalist task retention: RETAIN best preserves pretrained generalist performance relative to all fine-tuning baselines (Task-FT, Co-FT, LoRA, Freeze-FT, Scratch).
  • Modality ablation: Merging only the LLM backbone parameters is often sufficient; merging all components does not consistently improve results.
  • Scaling: Merging performance improves as the amount of pretraining data increases.

Limitations

  • \(\alpha\) is a tunable hyperparameter requiring at least one OOD validation scene, which may not always be available in practice.
  • Evaluated on two real-world tasks and three simulated tasks; breadth of evaluation is limited relative to the diversity of real-world deployment scenarios.
  • Pretraining data must be accessible for RETAIN-co-FT; when data is proprietary, only RETAIN-task-FT is applicable.
  • The continual merging scheme is sequential (not parallel), and the interaction between many merged tasks at large \(N\) is not deeply analyzed.
  • Merging coefficient sensitivity across different OOD domains is acknowledged but not fully characterized.

Relevance to Vision-Language Models

RETAIN directly targets VLA models—architectures that fuse a vision encoder, an LLM backbone, and an action decoder—making it a fine-tuning methodology for the action-grounded frontier of VLMs. The finding that merging primarily the LLM backbone is sufficient connects to the broader literature on model merging for vision-language models (WiSE-FT, TIES-Merging) and suggests the LLM component carries most of the generalizable representational knowledge. For VLM researchers, RETAIN demonstrates that weight interpolation—already validated in unimodal settings—transfers robustly to multimodal, embodied policy models, opening a path for mitigating distribution shift in any VLM fine-tuned on small domain-specific datasets. The continual merging framework is also directly relevant to lifelong adaptation of large VLMs without replay buffers.