Skip to content

Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Point-RFT is a two-stage multimodal reasoning framework that augments Chain-of-Thought (CoT) with pixel-level point grounding, requiring models to explicitly localize visual elements during reasoning. By first fine-tuning on a 71K grounded CoT dataset and then applying GRPO-based reinforcement learning, it lifts ChartQA accuracy from 70.88% (base) to 90.04%, outperforming text-only RL by 6.12 points absolute.

Problem

Text-only CoT is insufficient for vision-language reasoning tasks: models hallucinate non-existent visual elements, fail to correctly perceive chart values, and cannot disentangle perception errors from reasoning errors. Existing multimodal CoT work is predominantly prompt-driven rather than trained end-to-end with RL, limiting how far reasoning quality can be pushed.

Method

Stage 1 — Supervised Format Fine-tuning (SFT) on Point-CoT: A 71K-sample dataset is constructed by combining GPT-4o (step-by-step text rationales) with Molmo-7B (pixel coordinate annotation). Each reasoning step is linked to <points x=... y=...> tags identifying the precise image location of referenced visual elements. Cross-validation between GPT-4o's element count claims and Molmo-7B's grounded coordinates filters hallucinated steps. The base model (Qwen2.5-VL-7B) is fine-tuned on this dataset to produce interleaved grounded CoT in <think>...<answer> format.

Stage 2 — Reinforcement Fine-tuning (RFT) with GRPO: The SFT model is further optimized via Group-wise Relative Policy Optimization on 15K ChartQA training samples. The combined reward R̂ = Rf + Ra penalizes structural non-compliance (format reward via regex) and incorrect answers (accuracy reward via exact/soft match). KL divergence against the SFT policy regularizes exploration.

Key Contributions

  • First end-to-end framework combining visually grounded CoT with reinforcement fine-tuning for multimodal reasoning.
  • Point-CoT dataset: 71K image–question pairs with every reasoning step aligned to image-space coordinates, constructed via a GPT-4o + Molmo-7B hybrid pipeline with cross-validation filtering.
  • Empirical demonstration that grounded CoT outperforms text-only CoT under both SFT and RL regimes.
  • Interpretability benefit: explicit point annotations allow disentangling perception errors from reasoning errors.

Results

  • ChartQA (in-domain): Point-RFT 90.04% vs. Base-RFT (text-only RL) 83.92% (+6.12%), vs. Point-SFT 87.21%, vs. base Qwen2.5-VL-7B 70.88%.
  • OOD average (CharXiv, PlotQA, IconQA, TabMWP, Counting): Point-RFT 53.16% vs. Base-RFT 33.80% (+19.36%), vs. Point-SFT 46.78%.
  • Gains on smaller backbone (Qwen2.5-VL-3B): 86.24% ChartQA, up from base 79.28%.
  • Gains on Qwen2-VL-7B: 86.16% ChartQA, up from base 55.40%.
  • Removing visual grounding drops SFT accuracy by 29.1 pp (56.72% → 27.60%); RL by 3.88 pp.
  • XML coordinate format (56.72%) outperforms JSON (49.36%) for structured visual reasoning.
  • All improvements statistically significant at p < 0.01.

Limitations

  • RFT is performed only on ChartQA data; generalization is evaluated but not explicitly trained for other chart/document domains.
  • The approach relies on Molmo-7B for coordinate annotation quality; errors in grounding propagate to training data.
  • Dataset composition is heavily weighted toward chart/plot sources (ChartQA 41.5%, DVQA 35.5%); coverage of other visual document types is limited.
  • Broader impact and additional limitations are deferred to supplementary material not included in the paper text.
  • Cross-validation filtering (discarding mismatches after 8 retries) may systematically under-represent visually complex scenes where grounding is hard.

Relevance to Vision-Language Models

Point-RFT addresses a fundamental bottleneck in VLM reasoning: the disconnect between language-space CoT and visual perception. By making the model's intermediate reasoning steps spatially grounded via pixel coordinates, it operationalizes the idea that multimodal thinking should reference modality-specific evidence rather than relying purely on verbalized descriptions. This is directly relevant to the broader line of work on scaling inference-time compute for VLMs—it shows that the quality of the reasoning trace (grounded vs. text-only) matters as much as its length. The approach also contributes a diagnostic capability (perception vs. reasoning error disentanglement) useful for understanding VLM failure modes.