Never Too Late for Force: Accelerating VLA Post-Training with Reactive Force Injection

🕒 Published (v1): 2026-07-15 18:01 UTC · Source: Arxiv · link

Why this paper was selected

Force injection during VLA post-training; addresses contact-state blindspot in vision-driven policies

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LIFT (Late Reactive Injection of Force) is a post-training framework that grafts force sensing onto a pretrained VLA (specifically \(\pi_{0.5}\)) without discarding its vision-language prior. It injects 6D end-effector wrench history through a causal force memory and zero-initialized cross-attention into a copied reactive action expert, then closes the loop via online DAgger. Across three contact-rich manipulation tasks, LIFT reaches higher peak success rates with fewer collected samples than vision-only post-training baselines.

Problem

Pretrained VLA policies are largely vision-driven and fail in contact states where depth is ambiguous, the scene is occluded, or small force errors push execution off the offline demonstration distribution. Including force at pretraining scale is impractical due to hardware cost and platform dependency, so force must be added during post-training. Three sub-problems must be solved simultaneously: (1) how to inject high-frequency, low-dimensional force so the policy reacts within an action chunk; (2) how to preserve the pretrained prior when adding a new modality path; and (3) how to handle force distribution shift, since force patterns are highly policy-dependent and offline force data cannot cover on-policy contact states.

Method

LIFT builds on a pretrained \(\pi_{0.5}\) and adds a reactive action expert beside the original base action expert in a two-stage pipeline.

Stage 1 (vision-only alignment): LIFT trains on handheld-collected demonstrations \(\mathcal{D}_v = \{(\ell, I, a)\}\) with force masked out, aligning the model to target tasks while preserving the pretrained prior.

Stage 2 (force-aware online post-training): A Flexiv Rizon 4S robot with a 6D wrench sensor collects on-policy corrective data \(\mathcal{D}_f^{(k)} = \{(\ell, I, F, a^*)\}\) via human-corrected DAgger rollouts. The model trains on a 1:1 mixture (RLPD-style) of \(\mathcal{D}_v\) and \(\mathcal{D}_f\).

Architecture (O1): The reactive action expert decodes chunk actions causally (action-by-action) rather than with full within-chunk attention. A GRU wrench encoder produces a causal force memory \(m_{t:t+H}\), which is injected into the reactive branch through force-injected cross-attention with a latency-aligned causal mask. The expensive vision-language prefix is computed once and KV-cached; within each chunk, only the lightweight reactive branch re-runs against the cached prefix.

Initialization (O2): The reactive action expert is initialized by copying the base action expert weights. A shifted causal attention pattern ensures each reactive token \(r_i\) attends to the VL prefix, later base-action tokens \(a_{i+1:H-1}\), and causal reactive prefix \(r_{0:i}\) — making it output-equivalent to the original at initialization. The cross-attention output projection is zero-initialized, so the force residual is exactly zero before post-training begins.

Training (O3): A shared additive flow-matching objective trains both base and reactive streams: $\(\mathcal{L}(\theta) = \|v_\theta(x_\tau,\tau,o)-u\|^2 + \|v_\theta^r(x_\tau^r,\tau,o,m_{t:t+H})-u^r\|^2\)$ Vision-only samples use zero force placeholders and mask gradients to the force encoder; online samples activate the full force pathway. At inference, only the reactive stream's output is sent to the robot controller.

Key Contributions

• Reactive force injection via causal force memory (GRU-encoded 6D wrench history) and cached vision-language prefix, enabling within-chunk action refresh without full forward-pass latency.
• Prior-preserving initialization via weight copying into the reactive action expert, shifted causal attention for initialization equivalence, and zero-initialized cross-attention output projection.
• Force-aware online DAgger post-training with explicit force masking and 1:1 balanced sampling to jointly exploit heterogeneous offline visual and on-policy force-corrective data.
• Empirical demonstration that reactive force memory (multi-frame) outperforms single-frame force input on non-Markovian contact decisions (e.g., insertion phase detection).

Results

• Towel folding: LIFT peaks at 0.825 (2,800 samples) vs. \(\pi_{0.5}\) w/ Online DAgger peaking at 0.725 (3,100 samples). LIFT w/o Reactive Force Injection peaks at 0.95 (2,800 samples) on this task.
• Book insertion: LIFT peaks at 0.6 (4,600 samples) vs. 0.4 (5,600 samples) for vision-only DAgger; single-frame force baseline peaks at 0.4 (5,000–5,300 samples).
• Hanoi ring placement: LIFT reaches 0.6 (1,700 samples) and maintains 0.6 at final checkpoint vs. vision-only DAgger peaking at 0.3 (1,400 samples) and dropping to 0.2; single-frame force baseline collapses from 0.5 to 0.0–0.1 later in training.
• LIFT w/o Online DAgger underperforms on all tasks and drops to zero on book insertion, confirming online corrective data is necessary.
• Generalization evaluation under object, tablecloth, and lighting changes shows no clear performance drop in the final LIFT checkpoint across all three tasks.

Limitations

• Online DAgger requires human corrections during Stage 2, limiting data collection throughput and scalability.
• Evaluation is restricted to single-arm manipulation on one robot platform (Flexiv Rizon 4S).
• LIFT has not been tested across diverse robot arms, force sensors, or end-effector morphologies.
• Offline force data alone is insufficient (LIFT w/o Online DAgger fails), making the framework dependent on real-robot interaction infrastructure.

Relevance to Vision-Language Models

LIFT is directly relevant to VLA research — the dominant paradigm in robotics applications of VLMs — by addressing the fundamental limitation that large pretrained VLMs encode vision-language but not physical contact modalities. The zero-initialized cross-attention and weight-copying initialization strategy offers a general recipe for modality extension in post-training without catastrophic forgetting of the pretrained prior, a challenge shared broadly across VLM fine-tuning work. The online DAgger coupling highlights the distribution-shift problem endemic to deploying VLMs in closed-loop embodied settings. For researchers tracking VLMs, LIFT exemplifies a growing direction of bridging the gap between passive, web-scale pretraining and physically grounded deployment through targeted, modality-specific post-training.