Unleashing Perception-Time Scaling to Multimodal Reasoning Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Extends inference-time compute scaling to VLM perception; frontier capability paper

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Current Large Vision-Language Models (LVLMs) treat visual perception as a one-shot "fast" output, which prevents inference-time scaling (RLVR) from improving perceptual accuracy. This paper introduces Perception-Time Scaling (PTS), which reformulates perception as a structured, token-rich, decomposed process compatible with RL training. PTS improves high-precision visual estimation (RA\(_{0.1}\)) on the new DisTANCE benchmark from 8.0% to 64.7% on Qwen2.5-VL-7B.

Problem

Inference-time scaling via RLVR substantially improves reasoning in LVLMs but yields only marginal gains on visual perception tasks. The authors identify "fast perception" as the root cause: models express perceptual outputs in minimal tokens (e.g., "the radius is 2.5 units") without intermediate steps, so the chain-of-thought scaling that benefits reasoning never engages the perceptual process. Consequently, perception accuracy degrades as scene complexity grows, and reasoning-enhanced models (e.g., R1-OneVision, Vision-R1) match or underperform the base Qwen2.5-VL-7B on perception-centric tasks.

Method

Perception-Time Scaling (PTS) introduces two structural components applied to visual quantitative estimation:

1. Perception Elaboration: A symbolic encoding function \(\phi\) maps a target distance \(d_t\) into a discrete token sequence of <==========> groups, where each full group represents 1.0 unit and each = sign represents \(\delta = 0.1\) unit. Formally: $\(\phi(d_t) = \underbrace{\langle{=\cdots=}\rangle \cdots \langle{=\cdots=}\rangle}_{k} \| \phi_{\text{res}}(r), \quad k = \lfloor d_t \rfloor, \; r = d_t - k\)$ This forces the model to construct perception outputs incrementally rather than emit a single numerical token.

2. Perception Decomposition: Complex distance estimation is decomposed into iterative accumulation of a known reference segment \(d_r\) (length 1.0 unit): the model tiles \(d_r\) along \(d_t\) step-by-step (\(L \leftarrow L + d_r\), \(k \leftarrow k+1\) while \(L + d_r \leq d_t\)), mirroring ruler-based human measurement.

Training Pipeline (two stages): - Stage 1 — Cold Start (SFT): 6,000 PTS-style reasoning chains (2,000 per sub-task: length, perimeter, area) synthesized via GPT-4o few-shot prompting, covering a structured 5-step reasoning schema (Review → Hint → Reference → Estimation → Calculation). Fine-tuning teaches the PTS pattern. - Stage 2 — GRPO RL: The model is further optimized with GRPO using a continuous exponential reward: $\(r(o) = e^{-\alpha \frac{|o - d_t|}{d_t}}\)$ which is more sensitive to small relative errors than binary accuracy. A label normalization curriculum (normalized samples first, then full distribution) stabilizes early training.

Key Contributions

• DisTANCE benchmark: 300 synthetic image-question pairs for visual estimation (length, perimeter, area) using Relative Accuracy (RA) at multiple thresholds; exposes the perception gap across 20+ LVLMs.
• Fast Perception diagnosis: Quantifies that reasoning models extend output length but allocate only a small fraction to perception-related tokens, and that perception error scales with task complexity even for reasoning models.
• Perception-Time Scaling (PTS) paradigm: Symbolic elaboration + step-by-step decomposition that makes perceptual sub-steps explicit and RL-optimizable.
• Continuous regression reward with exponential sensitivity and label normalization curriculum for GRPO on non-binary tasks.
• Out-of-domain generalization: PTS gains transfer to Geoperception (LHC line comparison) and LEGO-Puzzles (Height) without in-domain data.
• Cross-task synergy: Mixing purely synthetic PTS data with math reasoning data yields consistent gains on both reasoning and real-world perception benchmarks.

Results

• DisTANCE (Qwen2.5-VL-7B + PTS SFT+RL): Average RA\(_{0.1}\) from 8.0% → 64.7%; average RA\(_\text{avg}\) from 21.5% → 88.3%.
• DisTANCE (Qwen2.5-VL-3B + PTS SFT+RL): Average RA\(_{0.1}\) from 6.3% → 37.7%; RA\(_\text{avg}\) from 19.5% → 73.5%.
• PTS vs. CoT (7B, SFT+RL): PTS RA\(_{0.1}\) 64.7% vs. CoT 37.7%; PTS RA\(_\text{avg}\) 88.3% vs. CoT 71.3%.
• PTS vs. Direct (7B, SFT+RL): PTS RA\(_{0.1}\) 64.7% vs. Direct 40.7%.
• Proprietary baselines on DisTANCE: GPT-5 RA\(_{0.1}\) = 17.0% (avg); Gemini-2.5-Pro = 21.0%—substantially below the trained PTS model.
• Out-of-domain (7B, SFT+RL): Geoperception LHC accuracy 59.8% → 78.7%; LEGO-Puzzles Height 30.0% → 33.0%; DisTANCE\(_\text{ood}\) perimeter RA\(_{0.1}\) 12% → 36%.
• PTS consistently outperforms spatial-aware models (Spatial-R1-7B: avg RA\(_{0.1}\) 9.3%) and tool-augmented LVLMs (GPT-4o + Sketchpad: 19.0%) on DisTANCE.

Limitations

• DisTANCE uses only synthetic geometric images; generalization to natural-scene perception (e.g., real-world depth, scale estimation) is evaluated only indirectly via LEGO-Puzzles and Geoperception, with modest gains.
• Symbolic token elaboration is hand-designed for distance; extending PTS to other perceptual modalities (color, texture, 3D pose) requires new encoding functions.
• PTS training data is limited to 6,000 SFT examples and restricted task types (length/perimeter/area); scaling to more diverse perception tasks is not demonstrated.
• Perception Decomposition as implemented assumes a known reference segment of defined length (1.0 unit) in the image; it is unclear how this generalizes when no such reference exists.
• The perception ratio analysis (Table 2) and attention analysis are conducted on a limited set of models; broader ablation of which component drives gains is incomplete.

Relevance to Vision-Language Models

This paper directly addresses a known failure mode of current VLMs: that inference-time scaling (RLVR) improves reasoning tokens but not perceptual grounding, potentially exacerbating hallucinations. The "fast perception" diagnosis—supported by token-fraction analysis and error-vs-complexity curves—provides a principled explanation for why reasoning-optimized LVLMs (R1-OneVision, MM-Eureka, etc.) fail to transfer reasoning gains to perception. The PTS paradigm is a practically actionable fix: it is model-agnostic (demonstrated on 3B and 7B Qwen2.5-VL), relies entirely on synthetic data, and is compatible with standard GRPO pipelines, making it readily applicable to any VLM trained with RLVR. The cross-task synergy finding—that PTS data improves math reasoning benchmarks as well—suggests that richer perceptual grounding may be mutually beneficial with reasoning, which is a key open question in the VLM training literature.