Skip to content

Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning

šŸ•’ Published (v1): 2025-01-01 Ā· Source: NeurIPS Ā· Venue: NeurIPS 2025 Ā· link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

šŸ’¬ Ask ChatGPTāœ¦ Ask Claude

TL;DR

Open Vision Reasoner (OVR) investigates how cognitive behaviors from language reasoning (backtracking, verification, subgoal setting, backward chaining) can be transferred to multimodal LLMs for visual reasoning. A two-stage pipelineā€”large-scale linguistic cold-start SFT on ~2M examples followed by ~900-step multimodal PPO RL on ~300K examplesā€”is applied to Qwen2.5-VL-7B, yielding state-of-the-art open-source results on visual math benchmarks. The work also provides a systematic behavioral analysis of when and how visual cognitive patterns emerge and scale across training stages.

Problem

Prior multimodal RL efforts either relied on RLHF-style learned reward models or constructed complex distillation pipelines for visual reasoning, without addressing the foundational mechanism by which linguistic cognitive behaviors (backtracking, verification, etc.) can transfer to and benefit visual reasoning in MLLMs. The conditions enabling this cross-modal cognitive transfer were poorly understood.

Method

Stage 1 ā€” Linguistic Cold Start: The LLM backbone of Qwen2.5-VL-7B is fine-tuned (vision encoder frozen) on ~2M language-only reasoning examples distilled from DeepSeek-R1, using batch size 640, sequence length 64k, lr 2Ɨ10ā»ā“, for 5 epochs. This instills cognitive behaviors (backtracking, subgoal decomposition) in a purely linguistic setting.

Stage 2 ā€” Multimodal RL: PPO with GAE (Ī³=1, Ī»=1) is applied under the Open-Reasoner-Zero framework for ~900 iterations on ~300K mixed-modality examples (text + image). Reward is binary (1/0) based on exact match of the predicted answer extracted from \boxed{}. Sequence length follows a curriculum: 24kā†’32kā†’48k tokens. The final model is a uniform checkpoint average over representative intermediate steps.

Cognitive Behavior Analysis: GPT-4o is used to detect the presence of four visual cognitive behaviors (visual reflection, divide-and-conquer, visual verification, goal-driven visual tracing) in OVR's inference traces, tracking emergence and language-to-vision transfer rates across training stages.

Key Contributions

  • Two-stage training paradigm (linguistic cold start + large-scale multimodal RLVR) establishing the largest open-source RL practice on Qwen2.5-VL-7B (~900 RL steps).
  • Empirical demonstration that cognitive behavior transfer from language to vision is real, early-emerging, and strategically selective (backtracking transfers readily; verification does not).
  • Identification of a cold-start/RL division of labor: cold start broadly memorizes behaviors, RL selectively amplifies high-utility ones (visual reflection, backtracking).
  • Analysis showing cold start degrades visual perception (hallucination increase) while multimodal RL partially recovers it, and that RL reward fails to scale on pure perceptual tasks (OCR, counting) due to absent response-length growth.
  • Full release of model, data, and training dynamics.

Results

  • AIME 2024: 63.5% (vs. best open-source 7B ReVisual-R1 at 53.3%; competitive with QwQ-32B-Preview at 50.0%)
  • AIME 2025: 52.1% (vs. Skywork-R1V-38B at 43.3%)
  • MATH500: 95.3% (vs. ReVisual-R1 at 89.2%, DeepSeek-R1-Distill-Qwen-7B at 92.8%)
  • MathVision: 51.8% ā€” first post-trained Qwen2.5-VL-7B model to exceed 50% (vs. ReVisual-R1 at 48.8%, VL-Rethinker at 28.4%)
  • MathVerse: 54.6% (vs. Skywork-R1V-38B at 54.6% at 38B scale, VL-Rethinker at 53.6%)
  • MMMU-Pro: +7.2% over prior SOTA among 7B models
  • GPQA Diamond: 49.8% (vs. ReVisual-R1 at 47.5%)
  • Perception benchmarks (MMBench): marginal gains from cold start (+0.8%) and RL (+0.5%) over base; HallusionBench degrades after cold start (49.0ā†’55.0 error rate), partially recovered by RL (53.6)

Limitations

  • Cold-start SFT on linguistic data causes perceptual degradation (increased hallucination, drops on POPE, RealWorldQA, MME) that is only partially mitigated by multimodal RL.
  • RL does not scale on discriminative perceptual tasks (OCR, counting): reward improves but response length stagnates, indicating absent core visual cognitive behaviors for these tasks.
  • Transfer mechanism is correlational; causal relationship between specific cognitive behaviors and downstream reasoning gains is not established.
  • Binary exact-match reward excludes proof-style and open-ended visual problems, limiting coverage of reasoning types.
  • Backtracking transfer is attributed to DeepSeek-R1's mental imagery in training data, making the approach dependent on the quality of the cold-start distillation source.

Relevance to Vision-Language Models

This paper directly addresses a core challenge in VLM training: how to endow MLLMs with the structured, self-correcting reasoning that makes LLMs powerful, without requiring task-specific reward engineering. By showing that linguistic cognitive behaviors (especially backtracking via mental imagery) transfer to visual modalities through standard RLVR, it validates a generalizable recipeā€”cold-start + rule-based RLā€”applicable to any capable VLM backbone. The behavioral analysis framework (tracking visual reflection, divide-and-conquer, etc. across training) provides concrete diagnostic tools for understanding and steering VLM reasoning development. The finding that RL selectively amplifies high-utility visual behaviors while suppressing ineffective ones has direct implications for curriculum and reward design in future VLM post-training work.