Skip to content

BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

BEAST is a training-free action tokenizer that encodes robot action sequences as B-spline control points, producing fixed-length discrete or continuous tokens compatible with parallel decoding. It integrates with VLMs (Florence-2) and simpler architectures, cutting inference latency to 19 ms and achieving state-of-the-art on CALVIN and competitive results on LIBERO without robot-data pretraining.

Problem

Existing action tokenizers for Vision-Language-Action (VLA) models suffer from at least one of: (1) requiring separate encoder-decoder training (VQ-BeT, Behavior Transformers); (2) producing variable-length token sequences that preclude parallel decoding (FAST/BPE); or (3) generating jerky trajectories due to discontinuities at action-chunk boundaries. Binning-based approaches (used in OpenVLA, RT-2) also have extremely low compression and fail on high-frequency continuous control.

Method

BEAST fits a clamped uniform B-spline to each action sequence via closed-form ridge regression (Cox–de Boor basis, least-squares solve: c = (Φ⊤Φ + λI)⁻¹Φ⊤a), extracting N control points per DoF. Control point matrices (shape D×N) are interleaved and quantized to uint8 [0,255] to form discrete tokens, or left as float for continuous use. Clamped B-splines guarantee the curve starts/ends at the boundary control points; smooth chunk-to-chunk transitions are enforced by pinning the first control point of each new chunk to the last action of the previous one and solving residual regression on the remaining points. No tokenizer training is needed. Three architecture variants are evaluated: BEAST-F (Florence-2 0.77B + parallel decoding via learnable action embeddings), BEAST-D (small decoder-only transformer trained from scratch), and BEAST-ACT (ACT CVAE predicting continuous B-spline control points).

Key Contributions

  • Training-free B-spline action tokenizer producing fixed-length tokens (no VQ codebook training, no BPE corpus).
  • Mathematically guaranteed smooth trajectory transitions across action chunks via clamped B-spline boundary conditions.
  • Parallel decoding compatibility (single forward pass for full action chunk) yielding 4–8× fewer tokens than binning.
  • BEAST-F VLA: Florence-2 (0.77B) + BEAST, no robot-data pretraining, competitive with 3.3B π₀.
  • Extensive evaluation: 166 simulated tasks (CALVIN, LIBERO, ALOHA) and 8 real-world tasks across 3 robot platforms.

Results

  • CALVIN ABC→D (Avg. Length, 1000 chains): BEAST-F 4.42 vs. MoDE 4.01, VPP 4.29, OpenVLA 3.27 — new SoTA without pretraining.
  • CALVIN ABCD→D: BEAST-F 4.61 vs. MDT 4.52, MoDE 4.39 — SoTA.
  • LIBERO average SR: BEAST-F 92.5% vs. OpenVLA 76.5%, π₀-FAST 85.5%; surpassed only by π₀ (94.2%, which uses large-scale pretraining). BEAST-F leads on LIBERO-Long (86.4% vs. π₀ 85.2%).
  • Inference throughput (RTX 4090, BF16): BEAST-F 617.3 Hz, latency 19 ms vs. π₀ 288 Hz/104 ms, Diffusion Policy 131 Hz/341 ms, OpenVLA 6 Hz/164 ms.
  • Real-world (8 tasks, 3 robots): BEAST-D 76.57%, BEAST-F 52.86%, π₀ 53.43%, FAST 28.5%.
  • Tokenizer ablation (LIBERO-Long / CALVIN): BEAST-PD 88/86% > BEAST-AR 84/84% > FAST-AR 71/84% > Binning-AR 46/28%.
  • Training efficiency: BEAST-F reaches ~80% on LIBERO-Long at 20k steps; π₀-FAST shows 0% at 30k steps without pretraining.
  • ALOHA bi-manual: BEAST-ACT 83%/23% (TransferCube/Insertion) vs. ACT 54%/6%.
  • N=10 basis functions optimal for 20-step chunks; MSE vs. binning: 0.0004 vs. 0.0215 (54× better).

Limitations

  • Performance is sensitive to the number of B-spline basis functions N; the optimal N depends on trajectory smoothness and sampling frequency and must be tuned per task/robot.
  • Experiments indicate 5–10 bases work for 1-second trajectories at typical robot frequencies, but no automated selection rule is provided beyond a heuristic in the appendix.
  • Continuous token variant (BEAST-CT) underperforms discrete tokens by 12.7% on CALVIN, attributed to difficulty modeling multimodal distributions with regression objectives.
  • Real-world dataset is small (50 demos/task), limiting generalization conclusions; larger-scale robot pretraining is left to future work.

Relevance to Vision-Language Models

BEAST directly addresses the action tokenization bottleneck in VLAs — models that ground VLMs in robot control — by replacing ad-hoc binning or learned VQ codebooks with a mathematically principled, training-free compression scheme. Its integration with Florence-2 (BEAST-F) shows that a sub-1B VLM can match or exceed 3.3B models (π₀, π₀-FAST) on manipulation benchmarks without robot-data pretraining, purely through better tokenization and parallel decoding. For researchers tracking VLMs, this work clarifies that action representation quality, not model scale or pretraining data volume, is a key bottleneck in VLA performance, and that classical signal-processing tools (B-splines) can outperform learned tokenizers while drastically reducing inference cost.