Skip to content

DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

RL incentivizes visual thinking in VLMs; novel image-based reasoning paradigm

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DeepEyes trains a 7B VLM to perform interleaved visual-textual reasoning ("thinking with images") purely via end-to-end reinforcement learning, without any cold-start supervised fine-tuning on reasoning trajectories. The model learns to autonomously decide when to crop and zoom into image regions during its chain-of-thought, forming an Interleaved Multimodal Chain-of-Thought (iMCoT) that emerges natively from the model's own grounding capability.

Problem

Existing VLMs reason almost entirely in the language modality, even when visual re-inspection would resolve ambiguity; they either hard-code visual-perception pipelines (SEAL, DyFo, ZoomEye) requiring expensive SFT data and task-specific design, or extend text-only CoT without actively querying the image. This decoupling of perception from reasoning causes failures on high-resolution, fine-grained, or spatially demanding tasks.

Method

DeepEyes wraps the model's native visual grounding capability as an internal zoom-in tool. Given question and image \(I_0\), the model may, after any textual CoT step, emit bounding-box coordinates; the framework crops those regions and appends the resulting image tokens \(I_{t_1}, I_{t_2}, \ldots\) to the rolling context. This produces an iMCoT where states are defined as \(s_t = \{X_{\le t}; I_{\le t}\}\) and actions sample the next token from the policy \(\pi_\theta\).

Training uses Group Relative Policy Optimization (GRPO) with a three-part reward: $\(R(\tau) = R_{\text{acc}}(\tau) + R_{\text{format}}(\tau) + \mathbb{1}_{R_{\text{acc}}(\tau)>0}\, R_{\text{tool}}(\tau)\)$ The conditional tool bonus \(R_{\text{tool}}\) is granted only when the trajectory is both correct and uses at least one zoom-in, preventing reward hacking via unnecessary actions. A perception-utility data filter pre-selects training samples that are (a) of intermediate difficulty and (b) verifiably solvable via active perception with known ground-truth regions, bootstrapping RL sampling efficiency without any SFT cold start.

Key Contributions

  • iMCoT framework enabling end-to-end RL training of interleaved visual-textual reasoning without cold-start SFT or external specialized models.
  • Active-perception data selection pipeline (difficulty filter + perception-utility filter) that ensures initial RL sampling efficiency.
  • Conditional tool reward design that encourages grounding-assisted reasoning while discouraging redundant zoom-ins.
  • Empirical characterization of three-stage RL training dynamics: (1) aimless exploration → (2) high-frequency over-querying → (3) selective, efficient exploitation.
  • Observation of diverse emergent reasoning patterns: visual search, visual comparison, visual confirmation, and hallucination mitigation via focused zooming.

Results

  • V* Bench (fine-grained, high-res): 90.1% overall (+18.9% vs. Qwen2.5-VL-7B; surpasses all non-end-to-end pipelines including ZoomEye at 90.6%).
  • HR-Bench-4K: +6.3% overall vs. Qwen2.5-VL-7B; HR-Bench-8K: +7.3%.
  • MME-RealWorld-Lite (general perception/reasoning): 53.2% overall (+10.9% vs. Qwen2.5-VL-7B), outperforming even the 32B Qwen2.5-VL.
  • Grounding (refCOCO/+/g, ReasonSeg): modest gains (+0.3–1.0 pts) over Qwen2.5-VL-7B.
  • POPE hallucination: +4.6% on Random split; +1.8% overall.
  • Math reasoning (MathVista/Verse/Vision, WeMath, DynaMath, LogicVista): consistent +1.0–4.3% gains over Qwen2.5-VL-7B.
  • 32B scaling: DeepEyes-32B reaches 93.3% on V* vs. 87.9% for Qwen2.5-VL-32B, with IoU 0.53 vs. 0.37 for 7B.

Limitations

  • Maximum of 6 active perception steps per trajectory is hard-capped during training, potentially limiting depth of visual exploration on very complex scenes.
  • The perception-utility filter relies on the existence of ground-truth bounding boxes for fine-grained perception data; this annotation requirement constrains which datasets can be used.
  • Grounding benchmark improvements are marginal (+0.3–1.0 pts on refCOCO) and one metric (POPE Adversarial) regresses by 1.9 pts vs. Qwen2.5-VL-7B.
  • KL coefficient is set to 0.0, removing explicit policy-drift regularization; stability implications for longer training runs are not analyzed.
  • Generalization to video, 3D, or multi-image settings is not evaluated in the main paper.

Relevance to Vision-Language Models

DeepEyes directly advances the VLM reasoning frontier by showing that image-conditioned chain-of-thought—where visual re-querying is an integral reasoning action rather than a pre-specified pipeline step—can emerge from pure outcome-supervised RL. This is a direct open-source analog to what OpenAI o3 reportedly does, making the mechanism transparent and reproducible. For researchers tracking VLMs, the paper demonstrates that reinforcement learning alone (without multimodal SFT trajectories) is sufficient to unlock test-time visual compute scaling, a result that connects to the broader trend of RL-based reasoning in LLMs (DeepSeek-R1, etc.) being productively extended to the multimodal domain. The conditional tool reward design and three-stage training dynamics offer concrete design lessons for training future agentic VLMs.