Skip to content

Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculation

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; speculative drafting approach for information-intensive visual reasoning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Speculative Verdict (SV) is a training-free framework that borrows the draft-then-verify paradigm from speculative decoding and applies it to information-intensive visual reasoning. Multiple lightweight VLMs generate diverse reasoning paths (drafts), and a single large VLM synthesizes them into a final answer (verdict). SV consistently outperforms both the underlying draft models and large proprietary baselines like GPT-4o on chart and infographic VQA benchmarks.

Problem

Current VLMs fail on information-intensive images (infographics, dense charts) that require two coupled capabilities: (1) precise localization of query-relevant cues across dense, multi-scale layouts, and (2) multi-hop reasoning that chains visual evidence (colors, shapes, spatial relations) with textual annotations. Existing approaches—attention-based zoom-in cropping and RL-trained zoom policies—either track irrelevant regions due to weak signal-evidence correlation or require costly task-specific supervised training.

Method

SV operates in two stages without any fine-tuning:

Draft stage. A pool of \(k=5\) small VLMs (7–9B parameters) first generate candidate answers. A consensus score \(s(y_i)\) is computed for each candidate by summing the normalized negative log-likelihood that each peer model \(M_j\) assigns to answer \(y_i\): $\(s(y_i) = \sum_{j \neq i} \left[ \text{NLL}_j(y_i) - \text{NLL}_j(y_j) \right]\)$ The \(m=3\) candidates with the lowest (most consensus) scores are selected as draft experts. Each expert is then prompted with Chain-of-Thought to generate a full reasoning path covering localization proposals, evidence extraction, and analytic operations.

Verdict stage. A single large VLM (GPT-4o or Qwen2.5-VL-72B) receives the original image plus all \(m\) reasoning paths concatenated as context and produces the final answer in one inference call. The large model acts as a synthesizer—validating grounding, resolving contradictions, and integrating consistent cues—rather than a voter. Computation is concentrated in the prefill of long reasoning paths, keeping decoding costs low.

Key Contributions

  • Training-free adaptation of speculative decoding to multimodal reasoning for error correction rather than speed.
  • Consensus expert selection via peer NLL scoring that is efficient (no extra decoding) and selects the most reliable draft reasoning paths.
  • Empirical demonstration that the verdict can recover correct answers from minority-correct and even zero-correct draft configurations (47–53% and 2.5–4.5% recovery rates, respectively).
  • Outperforms RL-trained zoom-in pipelines (DeepEyes, Pixel-Reasoner) without task-specific training.

Results

  • InfographicVQA (ANLS): SV w/ GPT-4o = 88.4 (+11.9 vs. GPT-4o alone, +4.2 vs. best open-source 84.8); SV w/ Qwen2.5-VL-72B = 86.7 (+2.5 vs. that verdict alone).
  • ChartMuseum (Acc): SV w/ GPT-4o = 49.3 (+6.6 vs. GPT-4o 42.7); SV w/ Qwen2.5-VL-72B = 48.2 (+7.5).
  • ChartQAPro (Acc): SV w/ GPT-4o = 64.0 (+11.4 vs. GPT-4o 52.6); SV w/ Qwen2.5-VL-72B = 63.0 (+2.3).
  • HR-Bench 4K (Acc): SV w/ Qwen2.5-VL-72B = 75.6 (+2.5); SV w/ GPT-4o = 71.4 (+4.0 vs. GPT-4o 67.4).
  • MathVista (Acc): SV w/ GPT-4o = 82.9 (+17.8 vs. GPT-4o 65.1).
  • TallyQA-Complex (Acc): SV w/ GPT-4o = 76.9 (+1.5 vs. GPT-4o 75.4).
  • SV improves DeepEyes by 12.9% on InfographicVQA.
  • Error correction: SV recovers 47–53% of minority-correct cases and 2.5–4.5% of zero-correct cases where the verdict alone fails.

Limitations

  • Requires running \(k\) candidate VLMs plus one large verdict model; total wall-clock cost is higher than a single-model call, even if decoding cost for the verdict is reduced.
  • The draft pool is fixed and hand-curated (5 specific 7–9B models); generalization to other draft configurations is not fully explored.
  • Image preprocessing with PP-StructureV3 is applied as an auxiliary step for information-intensive benchmarks, adding a pipeline dependency.
  • Consensus scoring assumes that candidate answers are textually comparable; it may degrade when correct answers are expressed in diverse surface forms.
  • Gains on counting-heavy (TallyQA) and high-resolution (HR-Bench) benchmarks are smaller than on chart-intensive tasks, suggesting diminishing returns outside the core information-intensive setting.

Relevance to Vision-Language Models

SV directly addresses a well-known weak point of current VLMs—dense, multi-scale information retrieval from charts and infographics—without requiring any model training, making it immediately applicable to off-the-shelf models. The consensus-based expert selection is a lightweight, model-agnostic mechanism that could generalize to other multi-VLM ensemble pipelines. By repurposing speculative decoding from a speed optimization into an accuracy and error-correction tool, the work opens a new design axis for inference-time reasoning in multimodal systems. For researchers tracking VLMs, this paper is representative of the growing "test-time compute" trend applied to the multimodal domain.