Skip to content

No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Georgia Gkioxari/Meta; label-free visual reasoning training via multimodal verifiers

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VALOR is an annotation-free training framework for visual spatial reasoning that jointly tunes an LLM (for program-based task decomposition and tool use) and a visual grounding model (GroundingDINO) using multimodal verifiers as the sole learning signal. An LLM verifier (Gemini-2.5-Flash) drives RL over a six-component reward model, while a VLM verifier pipeline generates pseudo-labels for grounding fine-tuning via hard-negative mining. Without any ground truth labels, VALOR outperforms open-source LLMs and matches or exceeds several proprietary models across eight spatial reasoning benchmarks.

Problem

Existing visual reasoning methods bifurcate into two failing camps: (1) language-only chain-of-thought VLMs that require large-scale \((image, query, answer)\) supervision and exhibit weak grounding and perceptual errors (e.g., confusing pixel-wise size with real-world 3D size), and (2) program synthesis approaches that invoke pre-trained vision specialists without training but suffer from faulty logical decomposition and poorly domain-aligned grounding models. Neither camp jointly addresses reasoning quality and grounding quality in an annotation-free manner.

Method

VALOR generates a natural language plan and Python program from a query \(q\) using a base LLM \(\pi_\theta\) (Qwen3-8B), with access to three vision specialist APIs: GD_DETECT (GroundingDINO for bounding boxes), DEPTH (MoGe2 for metric depth), and VQA (GPT-5-mini on image crops).

LLM tuning via RL. A composite reward \(R\) is defined as: $\(R(q,p,c) = r_\text{fmt}(p,c) \cdot \left[\lambda_\text{sn}\,r_\text{sn}(c) + \lambda_\text{log}\,r_\text{log}(q,p) + \lambda_\text{att}\,r_\text{att}(q,p) + \lambda_\text{sp}\,r_\text{sp}(q,p) + \lambda_\text{ad}\,r_\text{ad}(p,c)\right]\)$ with six binary components: format, syntax, logic coherence, attribute identification, spatial relationship coverage, and plan-to-code adherence. Format is a hard-gating multiplier; components (3)–(6) are evaluated by a frozen Gemini-2.5-Flash LLM verifier with task-specific prompts. The LLM is optimized with GRPO on 800 unlabeled \((image, query)\) pairs (400 from SA-1B with Gemini-generated queries, 400 query-only from Omni3D-Bench).

Grounding tuning via VLM verifiers. From generated programs, grounding calls (e.g., GD_DETECT("helmet")) are parsed and executed with lowered confidence threshold to over-predict boxes. A frozen VLM verifier then prunes via three stages: (1) coarse filtering over the full image with overlaid boxes, (2) per-crop verification on individual detections, and (3) deduplication. Surviving boxes become pseudo-labels for fine-tuning GroundingDINO (frozen vision/language encoders, trained on 4×A100 for 7 epochs). The full pipeline yields 30,826 bounding box annotations from 7,373 images with no human annotation.

Key Contributions

  • Annotation-free training paradigm (VALOR) that jointly improves LLM reasoning and visual grounding via multimodal verifiers.
  • Six-component reward model decomposing program quality for spatial reasoning; format acts as a hard constraint gate.
  • Three-stage VLM hard-negative mining pipeline for pseudo-label generation for object detection fine-tuning.
  • Empirical demonstration that verifier-quality matters critically: Gemini-2.5-Flash achieves 87% agreement with human annotations vs. 15% (Qwen3-8B) and 7% (Llama-3.2-11B-Instruct).
  • Scaling analysis showing upward trends in reasoning and execution accuracy with more unlabeled training data.

Results

All numbers from Table 1 (exact-match accuracy / MRA):

  • Omni3D-Bench (3D spatial): VALOR 44.0 vs. Qwen3-8B 37.5, Llama-3.2-11B 32.0, GPT-4o 38.0, Gemini-2.5-Flash 37.1.
  • RoboSpatial: VALOR 69.5 vs. Qwen3-8B 60.5, o4-mini 61.8, Gemini-2.5-Flash 68.7.
  • BLINK: VALOR 69.2 vs. Qwen3-8B 63.9, GPT-4o 64.2.
  • VSR: VALOR 75.6 vs. Qwen3-8B 68.2, o4-mini 68.5.
  • RealWorldQA: VALOR 57.3 vs. Qwen3-8B 53.3, o4-mini 65.9.
  • GQA: VALOR 64.4 vs. Qwen3-8B 57.4, GPT-4o 58.0.
  • CountBenchQA: VALOR 75.9 vs. Qwen3-8B 68.6, GPT-4o 67.6.
  • VLM verifier pipeline precision: 0.45 (coarse) → 0.50 (per-crop) → 0.75 (after deduplication); deduplication contributes the largest single gain (+0.25).
  • VALOR-RL (reasoning only, without improved grounding) already outperforms Qwen3-8B on 6 of 8 benchmarks; adding improved grounding (VALOR) yields further gains, especially on CountBenchQA (+8.3) and RoboSpatial (+7.7).

Limitations

  • Depends critically on a high-capacity proprietary verifier (Gemini-2.5-Flash); open-source alternatives degrade performance drastically, limiting fully open reproducibility.
  • VLM verification pipeline reaches only 75% precision, meaning 25% of pseudo-labels are false positives that introduce noise into grounding fine-tuning.
  • Pseudo-3D height estimation (\(h_\text{3D} = h_\text{2D} \times d\)) is a rough approximation that conflates metric depth with scale; accuracy depends on MoGe2 depth quality.
  • VQA specialist (GPT-5-mini) remains proprietary, creating a dependency in the full system.
  • Training corpus is small (800 pairs for RL), limiting coverage of edge-case query types; the paper notes this is intentional for GRPO stability but may cap generalization.

Relevance to Vision-Language Models

VALOR demonstrates a practical path for augmenting VLM-adjacent systems (grounding detectors, reasoning LLMs) without labeled data, using VLMs themselves as verifier-critics—a paradigm directly relevant to researchers building scalable VLM pipelines where annotation cost is prohibitive. The RL-with-verifiable-rewards transfer from text-only math reasoning (GRPO) to multimodal spatial reasoning is a key methodological bridge for the VLM community. The paper's finding that current frontier VLMs (GPT-4o, Gemini-2.5-Flash) fail at 3D spatial reasoning even with pixel-level reasoning traces—and that an 8B tool-use model can match or exceed them—has direct implications for evaluating VLM spatial understanding. The VLM-as-critic pipeline for grounding pseudo-labeling is a transferable technique for any domain requiring fine-tuned detectors without specialist annotators.