Skip to content

Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views

🕒 Published (v1): 2026-06-22 16:28 UTC · Source: Arxiv · Venue: ECCV 2026 · link

Why this paper was selected

ECCV 2026; dense reward shaping for multi-view 3D VQA—novel RL signal for spatial chain-of-thought

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DR-MV3D introduces a map-grounded multi-view 3D VQA framework that replaces sparse answer-level supervision with dense, verifiable intermediate rewards derived from frozen 3D vision foundation models. It decomposes reasoning into allocentric map construction, question-conditioned view-trajectory planning, and egocentric grounding, then optimizes the full pipeline with GRPO. A 3B model trained this way achieves 66.5% on MindCube-Tiny, a +28.7pp gain over the base Qwen2.5-VL-3B.

Problem

Multimodal LLMs trained with answer-level (sparse) rewards fail at multi-view 3D VQA: they produce geometrically inconsistent cognitive maps, select uninformative viewpoints, and misread egocentric directions. The core issue is that answer correctness provides no gradient signal for the intermediate spatial reasoning steps (map quality, view order), leading to brittle behavior under partial observability and compositional spatial queries.

Method

DR-MV3D builds on Qwen2.5-VL-3B-Instruct with SFT followed by GRPO. The reasoning trajectory has four stages:

  1. Global allocentric map \(\mathcal{C}^{\text{allo}}\): the MLLM generates a scene-level cognitive map from all input views. A frozen VFM pipeline (VGGT + SAM3) produces a geometry-consistent pseudo-ground-truth \(\mathcal{C}^*\), and a global consistency reward \(R_{\text{global}} = \texttt{sim}(\mathcal{C}^{\text{allo}}, \mathcal{C}^*)\) penalizes geometric deviation without manual annotation.

  2. View-trajectory planning: conditioned on \((\mathcal{C}^{\text{allo}}, q)\), the model autoregressively selects an ordered view sequence \(V = (v_1,\ldots,v_T)\). A local trajectory reward \(R_{\text{local}} = \frac{1}{T}\sum_t \mathbb{I}[v_t = v_t^*]\) measures step-wise match against a reference trajectory derived from benchmark metadata.

  3. Egocentric grounding: each selected view's global evidence is converted into an ego-aligned cognitive map \(\mathcal{C}^{\text{ego}}_t\) via coordinate rotation and reference-frame alignment, matching the MLLM's pretraining distribution.

  4. GRPO optimization: the composite reward \(R(\tau) = \lambda_g R_{\text{global}} + \lambda_l R_{\text{local}} + \lambda_a R_{\text{ans}} + \lambda_f R_{\text{fmt}}\) is used to compute group-relative advantages; the clipped GRPO objective with KL regularization updates \(\theta\) without a separate critic network.

Key Contributions

  • Dense intermediate reward scheme for MV3D-VQA: global map consistency reward (VFM-derived pseudo-GT, no human annotation) + local trajectory reward (step-wise viewpoint supervision).
  • Explicit allocentric-to-egocentric representation bridge that resolves the reference-frame mismatch inherent in MLLM pretraining.
  • Full GRPO pipeline over a 3B MLLM that outperforms prior map-based RL baselines trained on the same backbone family.
  • Evaluation across three complementary benchmarks (MindCube, VSI-Bench, BLINK MV) demonstrating consistent gains.

Results

  • MindCube-Tiny: 66.5% (GRPO) vs. 37.8% (Qwen2.5-VL-3B-Instruct base) and 62.4% (SFT only); +28.7pp over base; surpasses MindCube-CGMap-SFT (54.4%) and MindCube-CGMap-FFR-RL (53.7%).
  • Sub-category gains most pronounced on compositional tasks: Among 71.3% and Around 73.6% with GRPO vs. significantly lower instruct baseline scores.
  • VSI-Bench: 37.1 Avg (competitive among open-weight models).
  • BLINK (MV): 56.4 accuracy.

Limitations

  • Reference trajectory \(V^*\) is deterministically derived from benchmark metadata — this signal may not generalize to settings without such structured annotations.
  • Pseudo-GT cognitive maps depend on VGGT + SAM3 quality; failure modes of the frozen VFMs propagate into the global reward signal.
  • Evaluated on a 3B parameter model; scaling behavior and compute cost at larger scales are not reported.
  • View trajectory reward uses exact step-wise matching, penalizing semantically equivalent orderings that differ positionally.

Relevance to Vision-Language Models

This work directly addresses a known brittleness of VLMs/MLLMs in 3D spatial reasoning under partial observability, a failure mode well-documented on multi-view benchmarks. The use of frozen 3D VFMs (VGGT+SAM3) as annotation-free supervisory signals is a practically important pattern for grounding language-driven representations in geometry without curated data. The dense-reward GRPO training regime generalizes the DeepSeek-R1/R1-VL verifiable-reward paradigm from single-image 2D tasks to the multi-step, multi-view spatial domain, offering a template for process-level supervision in any VLM reasoning task with decomposable intermediate steps. For VLM researchers, the allocentric-to-egocentric grounding bridge also highlights a systematic representation misalignment that affects spatially-aware VLMs more broadly.