Skip to content

Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement

🕒 Published (v1): 2026-07-07 15:10 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ActionCache is a training-free, plug-and-play external cache that accelerates the iterative flow-matching action head of Vision-Language-Action (VLA) models by retrieving and reusing past intermediate action chunks as warm-start initializations. By indexing cached actions with compact multimodal keys derived from VLM output embeddings, it enables cross-timestep, cross-episode, and cross-task action reuse. It achieves up to 11.75× and 34.43× action-head speedups for π0.5 and GR00T-N1.6, respectively, with minimal task success rate degradation.

Problem

Flow-matching-based VLA models require multi-step iterative integration of a velocity field \(V_\theta\) to generate each action chunk, with the action head accounting for 37–66% of end-to-end inference latency. Existing plug-and-play acceleration methods (e.g., EfficientVLA) reduce per-step cost but cannot eliminate the need for multi-step integration. Learned warm-start methods (STEP, SDP) exploit only local temporal continuity and require retraining or additional modules, precluding true plug-and-play deployment on pretrained VLAs.

Method

At each control timestep \(t\), the VLM backbone produces context \(c_t\) and output embeddings \(h_t \in \mathbb{R}^D\). ActionCache compresses \(h_t\) into a compact key \(k_t \in \mathbb{R}^d\) via a fixed sparse ternary random projection: $\(k_t = Rh_t, \quad R = (r_{ij}) \in \{-1, 0, 1\}^{d \times D}\)$ where each row has \(pD/2\) entries of \(+1\), \(pD/2\) of \(-1\), and the rest zero (\(d=500\), \(p=0.01\)). Top-1 cosine similarity retrieves the nearest cached entry. If the similarity exceeds threshold \(T_\text{hit}\), the cached intermediate action chunk \(A^{\tau}_t\) (stored at denoising step \(N - N_\text{hit}\)) is used to warm-start flow integration, requiring only \(N_\text{hit}\) additional NFE instead of \(N\). At \(N_\text{hit}=0\), the cached action is executed directly. On a cache miss, full-step generation proceeds from Gaussian noise and the resulting intermediate action is added to a pending buffer; on episode success it is committed to the cache. Cache replacement uses LFU (shown empirically superior to LRU under capacity constraints) and evicts entries once the capacity (default 3,000) is reached.

Key Contributions

  • ActionCache: a training-free, architecture-agnostic external cache that accelerates pretrained flow-based VLAs without modifying weights or adding learned modules.
  • Multimodal key design: sparse ternary random projection of VLM output embeddings into a compact \(d=500\) dimensional key, enabling efficient nearest-neighbor retrieval that encodes visual, linguistic, and (optionally) robot-state context.
  • Conservative hit/miss rule: a cosine-similarity threshold \(T_\text{hit}\) gates reuse, with automatic fallback to full generation to preserve robustness in unseen situations.
  • Cross-task action reuse: demonstrated that early-episode actions (e.g., reach-to-object) transfer across tasks, yielding >80% hit rate at episode start in cross-task zero-shot settings.
  • Empirical analysis of accuracy–latency–capacity trade-offs, replacement policies, and key feature ablations.

Results

  • VLABench (Ï€0.5): ActionCache at \(N_\text{hit}=1\) achieves 32.4% success rate vs. 38.8% (full, NFE=10) at 5.22× speedup; at \(N_\text{hit}=0\), 32.9% success at 11.75× speedup (latency 1.6 ms vs. 18.8 ms). Compared to EfficientVLA at NFE=1 (6.9% success, 6.6 ms), ActionCache at \(N_\text{hit}=0\) is both faster and substantially more accurate.
  • VLABench (GR00T-N1.6): ActionCache at \(N_\text{hit}=1\) achieves 22.3% success at 34.43× speedup (0.7 ms vs. 24.1 ms for full NFE=4).
  • Cross-task (VLABench, Ï€0.5): Cache hit rate exceeds 80% at episode start; progress scores on select_painting (20.0 vs. 21.0) and select_toy (51.7 vs. 50.5) match the base model while achieving 2.7–8.1% hit rates with a cache populated only from select_fruit.
  • LFU vs. LRU (cache size=100): LFU achieves higher hit rate than LRU, FIFO, and Random at all \(T_\text{hit}\) thresholds, approaching the oracle policy.
  • Real-world (Ï€0.5 on SO-101 arm, pick-and-place, 100 episodes): ActionCache with \(N_\text{hit}=1\), cache size=1,000 evaluated; specific numbers not shown in provided text excerpt.

Limitations

  • Performance depends on cache warm-up: the cache must be pre-populated with successful episodes before hits can occur; cold-start degrades to full-step generation.
  • \(T_\text{hit}\) and cache size are task-specific hyperparameters requiring per-task tuning.
  • GR00T-N1.6 keys require explicit concatenation of robot-state features because raw VLM embeddings omit proprioceptive information, adding architecture-specific handling.
  • Cross-task reuse is demonstrated only on semantically similar manipulation tasks with shared sub-goal structure; generalization to highly dissimilar tasks is not evaluated.
  • Success rate drops are non-trivial: Ï€0.5 loses ~6 percentage points at \(N_\text{hit}=1\) vs. full NFE, and GR00T-N1.6 loses ~11.7 points at the most aggressive setting.
  • Evaluation is limited to two VLA models and two simulation benchmarks (VLABench, LIBERO); broader generalization is not established.

Relevance to Vision-Language Models

ActionCache directly targets the inference bottleneck introduced when a VLM backbone is coupled with an iterative generative action head, a dominant architectural pattern in modern VLA systems built on top of large VLMs. The cache key design exploits VLM output embeddings as a rich multimodal retrieval signal, making the efficiency of the approach contingent on the representational quality of the underlying VLM. For researchers tracking VLMs, this work illustrates a concrete inference-time reuse strategy that generalizes diffusion-caching ideas from image generation to embodied, grounded settings where retrieval must be conditioned on vision, language, and robot state jointly. It also highlights how the semantic structure encoded by VLM representations enables cross-task generalization even in low-level action generation.