Skip to content

FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

FreqPolicy accelerates flow-matching visuomotor policies to single-step inference by imposing frequency-domain consistency constraints on action velocity vectors across flow timesteps. It treats action chunks as temporal signals rather than static vectors, enforcing spectral alignment via DCT and an adaptive per-frequency-band loss. Integrated into OpenVLA, it achieves 5× faster inference with no performance degradation on 40 LIBERO tasks, and 93.5 Hz real-world control.

Problem

Generative flow-matching and diffusion-based visuomotor policies require multi-step ODE integration at inference (e.g., 10 steps for π0, 4 for GR00T N1), creating latency incompatible with high-frequency real-time robot control. Prior acceleration methods (consistency distillation, distribution matching) were imported from image generation, which ignores the temporal coherence of action trajectories—action chunks are time-series signals, not independent samples.

Method

FreqPolicy adds two auxiliary training objectives on top of the standard conditional flow matching loss (Eq. 8):

  1. Frequency Consistency Constraint (Lfreq): For two arbitrary flow timesteps r, s ∈ [0,1], the predicted velocity vectors vθ(r, ar) and vθ(s, as) are projected into the frequency domain via type-II Discrete Cosine Transform (DCT). The ℓ2 distance between spectral coefficients F(vr) and F(vs) is minimized. A second term enforces trajectory-level consistency: starting from ar and as, both flows must converge to the same point at a future time u > s > r. This promotes straight flows in frequency space without requiring a pretrained teacher.

  2. Adaptive Frequency Component Loss: Inspired by Focal Loss, per-frequency-band weights wk are computed via a softmax over the squared spectral differences between vr and vs (Eq. 13). The final loss (Eq. 14) is a weighted sum of per-band ℓ2 spectral distances, dynamically emphasizing frequency bands with high discrepancy—corresponding to high-dynamic motion phases (contact-rich, skill transitions) over low-dynamic phases (reaching).

Total loss: Ltotal = Lfm + Lfreq. At inference, a single ODE step (NFE=1) is used. The policy backbone handles both 2D RGB and 3D point cloud inputs. For VLA integration, FreqPolicy replaces the action head of OpenVLA.

Key Contributions

  • First visuomotor policy to impose frequency-domain consistency constraints on flow matching, treating action chunks as temporal signals.
  • Frequency consistency objective (DCT-based spectral alignment of velocity vectors across flow timesteps) enabling one-step generation without a teacher model.
  • Adaptive frequency component loss with data-driven per-band weighting that handles non-stationary motion phases.
  • VLA integration: OpenVLA-FreqPolicy achieves 94.8% average success on LIBERO at 6.05 Hz (1 NFE), versus 68.1% at 0.32 Hz for OpenVLA-DP (50 NFE).
  • Real-world demonstration at 93.5 Hz inference on physical robot arms (Franka, UR).

Results

Robomimic (2D, NFE=1 one-step methods): - FreqPolicy: Lift 1.00 / Can 0.98 / Square 0.92 / Transport 0.90 / Tool Hang 0.85 - vs. ConsistencyPolicy: 1.00 / 0.98 / 0.92 / 0.78 / 0.70 - vs. Consistent-FM: 1.00 / 0.94 / 0.90 / 0.84 / 0.80 - vs. IMLE Policy: 1.00 / 0.98 / 0.82 / 0.90 / 0.81

MetaWorld + Adroit (3D, 53 tasks, NFE=1): - FreqPolicy average: 78.5% ± 2.6 vs. SDM: 74.8% ± 4.5, FlowPolicy: 77.2% ± 2.8, DP3: 76.1% ± 2.3

LIBERO VLA (OpenVLA backbone, 40 tasks): - OpenVLA-FreqPolicy (NFE=1): 94.8% avg, 6.05 Hz - OpenVLA-FlowMatching (NFE=10): 93.7%, 1.26 Hz - OpenVLA-DP (NFE=50): 68.1%, 0.32 Hz; Long suite: 87.6% vs. 11.8%

Real-world (3 long-horizon tasks, RTX 4090): - Task 1 (Fruit Sorting): FreqPolicy 70% @ 93.5 Hz vs. DP-10step 55% @ 19.8 Hz, FM-10step 60% @ 20.2 Hz

Ablation (Robomimic): Full adaptive frequency constraint (#6) outperforms vanilla FM (#1) by up to 12 pp on Tool Hang (0.88 vs. 0.76); spatial-only consistency (#2) gives intermediate gains.

Limitations

  • Frequency consistency formulation was validated only on flow matching; extension to diffusion-based policies not yet demonstrated.
  • Training requires two forward passes per update (to compute velocity vectors at two random timesteps), doubling compute vs. standard flow matching.
  • Real-world experiments are limited to 3 tasks on 2 robot platforms; generalization to diverse morphologies or contact-rich dexterous tasks is untested.
  • No ablation on the choice of DCT vs. other frequency transforms (DFT, wavelets).

Relevance to Vision-Language Models

FreqPolicy directly targets VLA model deployment: the LIBERO experiments integrate the method into OpenVLA, showing that frequency-consistent flow heads can replace diffusion heads for 5× faster inference without accuracy loss, which is a concrete path to real-time deployment of large VLA models currently bottlenecked by multi-step action generation. The work establishes that temporal structure of action trajectories—ignored by prior image-generation-derived acceleration methods—is exploitable through frequency-domain losses, suggesting a design principle applicable to any VLA with a generative action head (π0, GR00T N1, DITA). For researchers tracking VLMs applied to embodied AI, FreqPolicy provides a drop-in efficiency module that bridges the gap between VLA model quality and the latency requirements of physical robot control.