Skip to content

Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

🕒 Published (v1): 2026-03-26 17:59 UTC · Source: Arxiv · Venue: CVPR 2026 · link

Why this paper was selected

CVPR 2026; preference alignment personalizes VLA driving behavior to individual styles

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Drive My Way (DMW) is a Vision-Language-Action (VLA) framework that personalizes end-to-end autonomous driving by learning driver-specific embeddings from real behavioral data and adapting to short-term natural language instructions via reinforcement fine-tuning. It collects a novel 30-driver dataset (PDD) in CARLA and demonstrates improved style-instruction adaptation on the Bench2Drive benchmark while maintaining safety. User studies confirm that generated behaviors are recognizable as each individual driver's own style.

Problem

Existing end-to-end autonomous driving systems optimize for generic objectives or offer only coarse preset modes (e.g., "sport"/"comfort"/"eco"), failing to capture the implicit, long-term behavioral individuality of human drivers or respond to real-time natural language intent. Language-driven personalization methods (e.g., Talk2Drive/RAG-based LLM approaches) are limited to simple, low-interaction settings and do not model cumulative long-term habits.

Method

DMW builds on a SimLingo VLA backbone (InternVL2-1B: InternViT-300M-448px + Qwen2-0.5B) and adds two personalization layers:

Long-term preference encoding: A DeBERTaV3-based profile encoder \(f_p(\cdot)\) and a temporal route processor \(f_b(\cdot)\) (multi-head self-attention over a past trajectory window of length \(k\)) are jointly trained with an InfoNCE contrastive loss: $\(\mathcal{L}^m_t = -\log \frac{\exp(\text{sim}(z^m_p, z^m_{b,t})/\tau)}{\sum_{j=1}^{M} \exp(\text{sim}(z^j_p, z^m_{b,t})/\tau)}\)$ This aligns text profile embeddings \(z^m_p\) with behavioral trajectory embeddings \(z^m_{b,t}\) in a shared latent space \(\mathcal{Z}\).

Reinforcement fine-tuning via GRPO: A residual decoder (learnable residual query tokens → MLP → categorical head) outputs discrete speed and steering residuals \(a^\Delta_t\) added to the backbone's base action \(a^{\text{base}}_t\). Rewards are a weighted combination \(R = w_s R_{\text{safety}} + w_e R_{\text{efficiency}} + w_c R_{\text{comfort}}\), where weights \((w_s, w_e, w_c)\) and thresholds (TTC \(\beta_{\text{safety}}\), preferred speed \(v_{\text{pref}}\), comfort bounds \(\beta_{\text{lat}}, \beta_{\text{long}}\)) are dynamically inferred per instruction via GPT-5 and refined by expert review. To promote behavioral diversity, trajectory augmentation pairs the most dissimilar driver pair (\(u = \arg\min_x \text{sim}(z^m_p, z^x_p)\)) and scales actions by route-level statistics: \(\tilde{a}^m_t = \frac{\bar{a}^m}{\bar{a}^u} \cdot a^m_t\).

Key Contributions

  • DMW framework: integrates user embeddings (long-term) and natural language instructions (short-term) into a VLA policy via contrastive pre-training + GRPO reinforcement fine-tuning with a residual action decoder.
  • Personalized Driving Dataset (PDD): first multi-modal dataset capturing 30 real drivers across 20 CARLA scenarios (overtaking, merging, intersections, pedestrian crossings) with structured profiles, ego-states, and expert speed reference from PDM-Lite.
  • Style-aware reward adaptation: LLM-generated, expert-refined per-instruction reward weights that map natural language style commands to distinct safety/efficiency/comfort objectives.
  • Alignment Score (AS): a user-study metric based on clustering driver logs and measuring roll-out classification accuracy plus human similarity ratings (1–10 scale) for zero-shot preference alignment on in-distribution (25) and out-of-distribution (5) drivers.

Results

Closed-loop evaluation on Bench2Drive (collision terminates episode):

  • DMW vs. SimLingo (aggressive style): DMW achieves Driving Score 79.50 vs. 78.56; Efficiency 281.56 vs. 247.60; higher Comfort (21.62 vs. 18.61) and longer Travel Time (26.93 vs. 25.35 s) indicating style-differentiated behavior.
  • DMW vs. SimLingo (conservative style): DS 82.72 vs. 78.18; SR 71.56 vs. 65.56; Comfort 34.62 vs. 26.99; TT 47.38 vs. 33.02 s — wider behavioral spread across styles.
  • DMW-Vanilla vs. DMW: Style-aware rewards (DMW) produce larger behavioral differentiation across styles (Speed range 6.18–7.72 m/s vs. 7.66–7.86 m/s for DMW-Vanilla) while DMW-Vanilla achieves slightly higher DS/SR due to unconstrained optimization.
  • StyleDrive baseline: Consistently lower DS (75.68–77.02) and SR (60.89–62.13) than DMW across all styles.
  • User studies show DMW roll-outs are correctly classified into the target driver's cluster and rated higher than baselines on the 1–10 similarity scale; zero-shot generalization to 5 OOD drivers is demonstrated.

Limitations

  • Dataset scale is modest: 30 drivers, 20 scenarios in simulation (CARLA) only — real-world transfer not validated.
  • Reward weights are derived from GPT-5 plus human expert review, introducing a dependency on an external proprietary LLM and manual refinement, limiting full automation.
  • The paper is truncated before full quantitative results for long-term alignment (RQ1) and OOD generalization numbers are summarized qualitatively.
  • Backbone is a 1B-parameter model (InternVL2-1B); scalability to larger VLAs or diverse sensor modalities is untested.
  • Behavioral augmentation by scaling to the most-dissimilar driver is a heuristic that may not generalize to multi-modal action distributions.

Relevance to Vision-Language Models

DMW is directly relevant to VLM/VLA research as it adapts a vision-language backbone (InternVL2-1B) for a decision-making task requiring both visual scene understanding and natural language grounding, extending beyond passive recognition to personalized closed-loop control. The contrastive alignment between text profile embeddings and behavioral trajectory embeddings is a novel application of vision-language representation learning — connecting semantic identity descriptors to continuous action manifolds. The GRPO reinforcement fine-tuning approach mirrors RLHF/RLAIF trends in LLMs, applied here to ground VLA outputs in individualized human preferences rather than generic human feedback. For researchers tracking VLMs, DMW illustrates how instruction-following and preference alignment techniques from LLM post-training can be directly ported into embodied, action-producing architectures.