Skip to content

TIM-PRM: Verifying multimodal reasoning with Tool-Integrated PRM

🕒 Published (v1): 2025-11-28 09:01 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TIM-PRM transforms multimodal Process Reward Model (PRM) verification from passive scalar scoring into an active, agentic loop: the verifier explicitly plans what to check, queries an external VQA tool with open-ended questions (Independent Question Asking), and synthesizes the grounded tool response into a step-level verdict. An 8B model trained on synthesized tool-integrated trajectories beats open-source models up to 78B on VisualProcessBench.

Problem

Existing multimodal PRMs fail in two ways: (1) discriminative/scalar PRMs are black-box and lack fine-grained visual grounding, misidentifying perception errors 35% of the time; (2) generative verifiers rely solely on parametric knowledge and exhibit sycophancy—they validate hallucinated visual claims (e.g., "the graph is a parabola") because the hypothesis biases the model's own perception. Neither paradigm can actively "look" at the image to ground claims independently of the reasoning context.

Method

TIM-PRM models step verification as an MDP over a four-part trajectory \(\tau_t = (z_\text{plan}, z_\text{call}, z_\text{resp}, z_\text{ana})\):

  1. Explicit Planning (<planning>): the model identifies what evidence (visual, knowledge, logic) is needed.
  2. Independent Question Asking (<tool_call>): instead of asking "Is claim \(h\) true?", the model issues open-ended queries \(q\) about image \(I\), decoupling the tool response from the hypothesis — \(z_\text{resp} = \text{Tool}(I, q)\).
  3. Tool Response (<tool>): an external MLLM (Qwen3-VL-30B) executes the VQA query.
  4. Grounded Analysis (<analyze>) + Final Verdict (<verify>): the model compares \(z_\text{resp}\) against \(s_t\) to emit \(v_t \in \{\text{Correct}, \text{Neutral}, \text{Incorrect}\}\).

Training data (~13k samples) is synthesized by a teacher model (Qwen3-VL-30B-A3B-Instruct) and filtered via MCTS consistency. A weighted NLL loss

\[\mathcal{L}(\theta) = -\mathbb{E}_{\tau \in D^+}[\log P_\theta(\tau)] - w \cdot \mathbb{E}_{\tau \in D^-}[\log P_\theta(\tau)]\]

upweights incorrect trajectories (\(w=10\) optimal) to counter class imbalance. LoRA fine-tuning is applied on Qwen3-VL-2B/8B base checkpoints.

Key Contributions

  • Identification and taxonomy of four failure modes in existing multimodal PRMs (Perception Error 35.1%, First Step Bias 33.3%, Knowledge Error 14.0%, Final Step Bias 10.5%).
  • TIM-PRM: an agentic PRM framework with structured tool-call trajectories that decouple visual perception from textual reasoning.
  • Independent Question Asking mechanism that formulates open-ended queries to avoid hypothesis-confirmation bias in tool invocation.
  • Sample upweighting strategy (\(w > 1\)) to correct for the prevalence of correct steps in MCTS-derived training data.
  • State-of-the-art FISI (First Incorrect Step Identification) F1 of 26.4, a 165% improvement over VisualPRM-8B (9.9).

Results

  • Step-wise F1 (VisualProcessBench overall): TIM-PRM-8B 61.7 vs. VisualPRM-8B 55.9, MM-PRM-8B 55.5, Qwen2.5-VL-72B 60.5, InternVL2.5-78B 52.6, GPT-4o 62.3 (proprietary).
  • FISI F1: TIM-PRM-8B 26.4 vs. MM-PRM-8B 14.4, VisualPRM-8B 9.9, Qwen3-VL-8B 17.2.
  • Tool strength ablation: upgrading the VQA backend from Qwen3-VL-2B to Qwen3-VL-30B improves overall F1 from 58.6 → 60.3.
  • Upweighting ablation: \(w=10\) yields 60.3 vs. 56.7 at \(w=1\).
  • Tool call frequency: ~21.6% of correct steps and ~20.4% of incorrect steps invoke the tool, showing calibrated (not indiscriminate) tool use.

Limitations

  • The external VQA tool (Qwen3-VL-30B-A3B-Instruct) is itself a large MLLM; verification latency and compute cost are substantially higher than scalar PRMs.
  • Tool quality directly limits verifier quality (scaling law shown), so deployment requires access to a capable VQA backend.
  • Evaluation is confined to VisualProcessBench (multimodal math); generalization to other reasoning domains (code, science diagrams outside math) is untested.
  • MCTS-derived training labels carry biases (First/Final Step Bias) that the filtering pipeline only partially corrects.
  • The ask_questions tool is limited to VQA; richer tool types (calculators, symbolic solvers) are not explored.

Relevance to Harnesses / Meta-Harnesses

TIM-PRM is a concrete instance of an agentic verification harness: a structured loop in which a judge model orchestrates external tool calls (planning → invoke → ingest response → analyze) rather than rendering a single-pass verdict. The four-part trajectory schema \(\tau_t = (z_\text{plan}, z_\text{call}, z_\text{resp}, z_\text{ana})\) is directly analogous to the tool-dispatch loop found in meta-harness architectures, where a coordinator decides when and what to delegate to subordinate modules. The Independent Question Asking mechanism formalizes a key harness design principle: decouple the oracle from the hypothesis to prevent context contamination. The training data pipeline (teacher synthesis → MCTS filtering → upweighted SFT) also models how a meta-harness can bootstrap sub-verifier behavior from stronger orchestrators.