Skip to content

VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VLM-R³ equips MLLMs with dynamic, iterative visual grounding during chain-of-thought reasoning: the model decides when and where to crop sub-images, injects those crops back into the reasoning context, and continues reasoning over updated visual evidence. Training uses a curated interleaved dataset (VLIR) for cold-start SFT followed by Region-Conditioned GRPO (R-GRPO) that rewards valid region selection conditioned on final-answer correctness. The framework sets new SOTA on MathVista, ScienceQA, and vision-centric benchmarks at the 7B scale.

Problem

Existing CoT-based MLLMs perform only a single static visual grounding at the start of the reasoning chain. As the reasoning trace lengthens, attention to the original image degrades, causing failures on tasks that require iteratively revisiting specific image regions—e.g., sequential hypothesis verification, fine-grained OCR, or spatial relationship tracking. Prior work (e.g., Visual CoT) introduced bounding boxes into reasoning but limited this to one pre-defined crop with no linguistic interleaving and no mechanism to credit-assign which region selection actually helped.

Method

VLIR Dataset Construction: 11,810 interleaved image-text rationale sequences are built from GQA, TextVQA, DocVQA, InfographicsVQA, and VSR. Qwen2.5-VL-72B generates multi-step rationales with embedded {"bbox_2d": [x1,y1,x2,y2]} crop commands; GPT-4o handles OCR-heavy samples. Filtering enforces (1) semantic unit validity of each crop (a smaller VLM confirms a recognizable entity in the crop) and (2) logical coherence via DeepSeek V3.

Interactive Inference Pipeline: At test time, when the model emits a JSON bounding box command, the system intercepts generation, crops and zooms the region, encodes it into visual tokens, and appends them to the input sequence. The model resumes from this new state. Multiple crop-zoom iterations are permitted per sample.

R-GRPO Training: Builds on GRPO with a critical mask: policy gradients are computed only over text and bounding box command tokens—never over injected image tokens (which are environment actions, not model actions). The advantage is normalized within a group of M sampled trajectories. The composite reward has four terms: - r_acc (1 if final answer correct, else 0) - r_format (1 if <answer> tags correct, else 0) - r_region (0.5 per valid non-redundant bbox, only when r_acc=1, capped at 0.5/episode) - r_length (0.001 per reasoning character, capped at 0.25/episode)

Cold-start SFT on VLIR precedes R-GRPO to bootstrap the bounding box format and region selection behavior.

Key Contributions

  • VLIR: First dataset providing step-level supervision on interleaved visual region selection and textual justification (11,810 samples across 5 source benchmarks).
  • R-GRPO: A policy optimization variant that handles mixed model-generated/environment-injected token sequences by masking image-token gradients; introduces region validity reward conditioned on answer correctness.
  • VLM-R³ framework: End-to-end pipeline enabling iterative crop-zoom-reason loops during inference, with state-of-the-art results on diverse multimodal benchmarks at 7B scale.

Results

  • MathVista: 70.4% vs. 68.2% (Qwen2.5-VL-7B base); surpasses LLaVA-CoT-11B (54.8%), R1-onevision-7B (64.1%), Mulberry-7B (63.1%).
  • MathVision: 30.2% vs. 25.1% base (+5.1%).
  • ScienceQA: 87.9% vs. 73.6% base (+14.3%); surpasses all open-source reasoning models listed.
  • MMMU: 62.2% vs. 58.6% base.
  • DocVQA: 96.8% vs. 95.7% base.
  • HallusionBench: 62.0% vs. 61.3% base; surpasses Mulberry (54.1%).
  • MME: 2432 vs. 2347 base.
  • Vision-centric average (MMVP, V, HR-Bench-4k/8k, MME-Realworld): 70.1% vs. 63.5% base; outperforms InternVL3-38B (avg not reported but dominates on V and MME-Real) and LLaVA-OneVision-72B.
  • Ablation: removing interleaved CoT costs −12.5% ScienceQA, −2.8% MMMU; removing VLIR SFT costs −15.7% ScienceQA, −5.2% MMMU; removing R-GRPO costs −3.3% ScienceQA, −0.7% MathVista.
  • Grounding accuracy analysis: ScienceQA degrades from 87.9% at 90% bbox accuracy to 54.1% at 40%, confirming strong sensitivity to grounding quality.

Limitations

  • VLIR dataset is modest (11,810 samples) and heavily skewed toward single-crop scenarios (94% of images have one crop); multi-hop reasoning with 3+ crops is underrepresented.
  • The approach requires an interactive inference pipeline that intercepts generation mid-stream—incompatible with standard batched decode; inference latency increases with number of crop iterations.
  • Region validity reward is binary and capped, which may under-incentivize nuanced region prioritization in multi-crop scenarios.
  • Grounding quality is a hard dependency: performance degrades sharply below ~70% bbox accuracy, creating a potential failure mode when the model hallucinates bounding boxes.
  • Evaluations are primarily on English-language benchmarks; generalization to multilingual or non-standard document layouts is unstudied.

Relevance to Vision-Language Models

VLM-R³ addresses a fundamental architectural bottleneck in current MLLMs: the inability to re-attend to visual evidence mid-reasoning. By treating image cropping as a tool call within the reasoning chain and training with RL that credits region selection quality, it moves VLMs toward an active perception paradigm analogous to how humans re-examine images when uncertain. The R-GRPO training objective—masking injected environment tokens from policy gradients—is a technically non-trivial adaptation of GRPO to interleaved multimodal sequences that is directly applicable to other tool-augmented MLLM settings. The work complements and extends the line of visual CoT methods (LLaVA-CoT, Visual-RFT, MM-Verify) and directly competes with the emerging family of RL-trained reasoning VLMs (R1-onevision, Mulberry), demonstrating that dynamic grounding is a stronger inductive bias than purely text-structured reasoning chains.