Skip to content

AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AutoVLA is an end-to-end autonomous driving VLA model that integrates discrete physical action tokens directly into a pretrained VLM (Qwen2.5-VL-3B), unifying scene reasoning and trajectory planning in a single autoregressive model. It introduces GRPO-based reinforcement fine-tuning (RFT) to enable adaptive switching between fast (direct action) and slow (chain-of-thought) thinking modes, reducing unnecessary reasoning while improving planning quality.

Problem

Existing VLA/VLM approaches for autonomous driving suffer from: (1) physically infeasible trajectory outputs when waypoints are generated as text, or increased architectural complexity when intermediate representations are used; (2) inflexible reasoning strategies that either always reason (high latency) or never reason (poor performance on complex scenes), lacking principled adaptation between the two.

Method

AutoVLA extends Qwen2.5-VL-3B with a K-disk clustered action codebook (K=2048 tokens, each representing short-term ∆x, ∆y, ∆θ movement over 0.5s) appended to the LM vocabulary, framing trajectory prediction as next-token prediction. Training proceeds in two stages: 1. SFT: Mixed trajectory-only (fast thinking) and CoT reasoning (slow thinking) data, with per-sample weighting (λ_cot=40) that upweights reasoning examples; CoT annotations (~45.6k for nuPlan, ~7.2k for Waymo) are distilled from Qwen2.5-VL-72B using GT actions as grounding hints. 2. RFT via GRPO: A group of G candidate outputs is sampled; rewards combine a driving quality signal (PDMS for nuPlan, ADE for Waymo) with a CoT length penalty (r = r_Driving − λ_r · r_CoT), penalizing unnecessary slow thinking. LoRA adapters are trained for parameter efficiency.

Key Contributions

  • Physical action tokenization via K-disk clustering (K=2048) integrated directly into VLM vocabulary, enabling end-to-end autoregressive trajectory planning
  • Dual thinking mode SFT: trains fast (action-only) and slow (CoT+action) modes jointly with CoT-weighted loss
  • GRPO-based RFT with a composite reward (driving quality + CoT length penalty) that learns when to engage CoT reasoning
  • Automated reasoning annotation pipeline using Qwen2.5-VL-72B with GT action hints for scalable CoT dataset construction

Results

  • nuPlan (NAVSIM): AutoVLA post-RFT achieves PDMS 89.11 vs. 80.54 (one-shot SFT); best-of-N reaches 92.12, competitive with Centaur (92.10) and TrajHF (93.95)
  • RFT efficiency: 10.6% PDMS improvement and 66.8% runtime reduction (avg. 500 test scenarios) after RFT
  • Thinking mode latency: Fast thinking avg. 1.07s vs. slow thinking avg. 10.52s
  • CARLA Bench2Drive (closed-loop): Driving score 78.84, success rate 57.73%, outperforming Orion (77.74 / 54.62%)
  • Action tokenization ablation: K-disk (K=2048) ADE 0.0281m vs. FAST (DCT) 0.1014m and RT-1 0.0182m at same K; NAVSIM PDMS 80.54 vs. FAST 67.63
  • Physical action vs. text waypoints: PDMS 80.54 vs. 71.31; runtime 3.95s vs. 7.65s
  • Data scaling: CoT-augmented training surpasses action-only training at ≥100k samples on both nuPlan and nuScenes

Limitations

  • Near-real-time inference at ~1 Hz (not true real-time); remains highly GPU-dependent with large memory requirements
  • K=2048 codebook at 0.5s resolution may not cover all rare/extreme vehicle movements
  • RFT rewards are benchmark-specific (PDMS for nuPlan, ADE for Waymo), limiting cross-domain generalization of the reward design
  • CoT annotations sourced from a single teacher model (Qwen2.5-VL-72B), potentially inheriting its biases

Relevance to Vision-Language Models

AutoVLA demonstrates a concrete recipe for grounding VLM outputs in physically constrained action spaces by extending the token vocabulary with domain-specific codebook tokens, a pattern applicable beyond driving. The GRPO-based adaptive reasoning mechanism addresses a general VLM efficiency problem—when to invoke expensive chain-of-thought—using a verifiable reward signal rather than heuristics. The knowledge distillation pipeline (large VLM → structured CoT annotations → fine-tuned small VLM) is a reusable approach for injecting task-specific reasoning into compact deployable models. This work sits at the intersection of VLM post-training, tool-grounded reasoning, and embodied action generation, all active fronts in the broader VLM research agenda.