Skip to content

Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Praxis-VLM demonstrates that VLMs can acquire robust decision-making reasoning skills from text-only data via GRPO reinforcement learning, then transfer those skills to vision-grounded inference—bypassing the need for large paired image-text training datasets. A two-stage training pipeline (math cold-start → text-based decision-making RL) with an adaptive reward scheme outperforms SFT baselines, especially on out-of-domain benchmarks.

Problem

Current VLMs lack explicit multi-step reasoning for complex situational decision-making (robotics, autonomous driving, social navigation), and collecting large-scale paired image-text datasets annotated with reasoning chains is prohibitively expensive. Prior RL-based reasoning approaches for VLMs still depend on such paired multimodal data.

Method

The core insight—validated empirically—is that replacing visual inputs with textual captions yields comparable or better performance on decision-making benchmarks, suggesting reasoning can be decoupled from visual perception and learned from text.

Pipeline: 1. Data construction: GPT-4o synthesizes a 10K text-only QA dataset where visual situations are replaced by textual descriptions; multiple-choice questions require multi-step action selection. 2. Stage 1 – Math cold-start: GRPO training on Geometry3K (numerical reasoning) using an adaptive R1 reward (R_tag + R_format + R_accuracy) to instill <think>...</think><answer>...</answer> format without an SFT warm-up phase. 3. Stage 2 – Text-based decision RL: GRPO on the synthesized text decision dataset, using R_format + R_accuracy + R_len (length bonus to encourage deliberate longer chains). 4. Inference: Full VLM (vision encoder + LM) processes actual image-text inputs; reasoning skills transfer from text training to visual modality.

Only the language model components are updated during training; the vision encoder is frozen during training but active at inference.

Key Contributions

  • Empirical finding that text-situation inputs match or exceed image-situation inputs on VIVA and PCA-Bench, motivating text-driven training.
  • Praxis-VLM: a two-stage GRPO framework that learns decision-making reasoning entirely from text, transferring to multimodal inference.
  • Adaptive R1 reward design targeting format, accuracy, and reasoning length at appropriate training stages.
  • Demonstration that the SFT-then-RL paradigm can be skipped when training instruction-tuned VLMs directly with GRPO + adaptive rewards.
  • Analysis of four reasoning dimensions (situational analysis, action/consequence evaluation, safety/risk management, norm adherence) and three failure mode categories.

Results

  • VIVA: Praxis-VLM-3B: 79.03% vs. Qwen2.5-VL-3B baseline 76.61% and best SFT 77.42%; Praxis-VLM-7B: 84.03% vs. 80.97% baseline.
  • PCA-Bench: Praxis-VLM-7B: 60.25% vs. 46.37% baseline and 53.00% best SFT; 3B: 50.79% vs. 48.58%.
  • EgoNormia (OOD): Praxis-VLM-3B: 54.27% vs. 51.92% baseline, 35.06% w/ SFT, 28.34% w/ Reason SFT; 7B: 54.33% vs. 46.19% baseline and ~34% SFT variants—SFT methods degrade severely on OOD.
  • Two-stage vs. one-stage: cold-start initialization consistently improves OOD generalization, especially on 7B (54.33% vs. 49.57%).
  • Majority vote (8 samples): Praxis-VLM-7B achieves Pass@8 of 72.23% on EgoNormia vs. 66.04% for Reason SFT, and 77.92% vs. 82.33% on PCA-Bench.

Limitations

  • Training data is entirely synthetic (GPT-4o generated), which may introduce biases or limit coverage of rare real-world scenarios.
  • Inference-time truncation at 1,024 tokens causes failures for very long reasoning chains (Len5 bin shows accuracy drop).
  • "Overthinking" in the longest reasoning bins occasionally degrades performance.
  • Three recurring failure modes remain unresolved: misinterpretation of visual context, safety/prioritization misjudgment, and normative/ethical misalignment.
  • EgoNormia (video) is out-of-domain; the model was not trained on temporal/sequential inputs, limiting video reasoning.
  • Only evaluated on Qwen2.5-VL (3B and 7B); generalization to other VLM architectures is untested.

Relevance to Vision-Language Models

Praxis-VLM contributes a data-efficient RL-based training paradigm directly addressing the bottleneck of paired multimodal data scarcity—a central challenge in VLM fine-tuning. The finding that text-driven GRPO transfers to visual inference challenges the assumption that multimodal reasoning must be learned from multimodal data, with implications for how the community thinks about modality alignment and skill transfer. It connects to the broader thread of reasoning-capable VLMs (R1-style training, GRPO for vision) while offering a more practical path via text surrogates. The qualitative analysis of reasoning dimensions and failure modes provides concrete diagnostic vocabulary useful for future VLM decision-making research.