Skip to content

VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VisionThink is a reinforcement-learning-based paradigm for efficient VLMs that starts inference from a 1/4-resolution image and autonomously emits a special token to request the full-resolution image only when necessary. It uses a multi-turn GRPO training algorithm with an LLM-as-Judge reward and a threshold-based penalty to prevent collapse toward always requesting high resolution, achieving strong performance on OCR tasks while cutting inference time substantially on general VQA.

Problem

Existing efficient VLM methods prune or merge a fixed fraction of visual tokens using predetermined thresholds, applying the same reduction uniformly regardless of whether a given image-question pair actually needs fine-grained detail. This wastes compute on simple tasks while degrading OCR-heavy tasks. The core gap is the absence of per-sample, content-aware resolution decisions.

Method

VisionThink extends Qwen2.5-VL-7B-Instruct via multi-turn GRPO:

  1. Downsampled first pass: the model receives a 4× downsampled image and reasons in <think> tags. If information is sufficient, it answers directly; otherwise it emits a structured JSON function call (special token) to request the original-resolution image.
  2. Multi-turn GRPO (Eq. 3): standard GRPO is extended to multi-turn rollouts; loss is masked so only VLM-generated tokens (not tool-response tokens) contribute to the gradient.
  3. LLM-as-Judge: an external LLM compares the model's open-ended answer to the ground truth with a binary (0/1) reward, enabling RL on general VQA without rule-based exact matching.
  4. Reward function: R_overall = R_accuracy + R_format − P_control. The penalty P_control is threshold-conditioned: when low-resolution correct-answer probability r < 0.2, penalize direct answers (encourage upscaling); when r ≥ 0.2, penalize resize calls (discourage unnecessary upscaling). Format reward (0.5) enforces <think>, <answer>, and JSON call structure.
  5. Data preparation: 20K training samples selected by rolling out the base model 8× at both resolutions; 10K classified as low-resolution-sufficient and 10K as high-resolution-necessary, based on accuracy differentials.

Key Contributions

  • Sample-level adaptive resolution: first paradigm where the VLM itself decides per-sample whether to upscale, rather than applying a fixed token-retention ratio.
  • Multi-turn GRPO for VLMs: extension of GRPO to multi-turn visual tool-call settings with token-loss masking.
  • LLM-as-Judge for general VQA RL: enables RL training beyond structured visual math to open-ended VQA using semantic match evaluation.
  • Threshold-based penalty control: prevents model collapse to either always-upscale or always-direct-answer equilibria.
  • Compatibility: the sample-level paradigm is orthogonal to spatial token-pruning methods (FastV, SparseVLM, VisionZip) and can be composed with them.

Results

  • VisionThink retaining ~51.3% visual tokens on average outperforms SparseVLM (70% tokens) and FastV (70% tokens) on 9 benchmarks in aggregate (101.4% vs. 93.6% and 98.4% relative to Vanilla baseline).
  • On OCR-sensitive benchmarks: ChartQA 79.8 (+9.0 pp vs. FastV at 50% retention), OCRBench 80.8 (+8.3 pp vs. SparseVLM at 50% retention).
  • MME score 2400, surpassing all closed-source models tested (GPT-4o: 2329, Claude-3.5 Sonnet: 1920, Gemini-1.5-Pro: N/A).
  • MathVerse 48.0 (+3.7 pp over Qwen2.5-VL-7B base), MMVet 67.1 (+8.9 pp over base).
  • Inference time vs. QwenRL (full-res): >2× faster on DocVQA (1109 s vs. 3076 s total); ~30% faster on MME and POPE.
  • On non-OCR benchmarks (MME, DocVQA), 70–79% of samples are answered at low resolution without upscaling; on OCR benchmarks (ChartQA, OCRBench), 62–93% request high resolution.

Limitations

  • Fixed 2× upscaling only; flexible multi-scale resolution hierarchies are not explored.
  • Capped at two-turn conversations; complex visual problems may benefit from >5 turns of tool calls.
  • Token reduction is achieved via image resizing only; more advanced spatial-level pruning within a resolution is not integrated.
  • No cold-start phase: zero-shot special-token generation must be reliable from the start, requiring careful prompt selection; its absence causes early-training performance drops.
  • Evaluation limited to a 7B-parameter model fine-tuned from Qwen2.5-VL; generalization to other architectures or scales is not demonstrated.

Relevance to Vision-Language Models

VisionThink directly addresses the visual token bottleneck that constrains VLM deployment on resource-limited hardware, shifting the efficiency locus from heuristic spatial pruning to learned, content-aware resolution routing. The LLM-as-Judge RL recipe extends the DeepSeek-R1/GRPO training paradigm—previously confined to visual math—to general open-ended VQA, which is a significant generalization for the RL-for-VLM research thread. The multi-turn tool-call framing establishes a template for VLMs that can dynamically query higher-fidelity inputs, relevant to agentic and document-understanding pipelines. For researchers tracking VLMs, this work links efficient inference, RL fine-tuning, and adaptive computation into a single coherent framework built on top of a competitive open-source base model.