Skip to content

Learning Multi-View Spatial Reasoning from Cross-View Relations

🕒 Published (v1): 2026-03-30 02:42 UTC · Source: Arxiv · Venue: CVPR 2026 · link

Why this paper was selected

CVPR 2026; cross-view relational learning for multi-view spatial reasoning in VLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

XVR is a 100K-sample multi-view VQA dataset that provides explicit cross-view geometric supervision across three task categories: Correspondence, Verification, and Localization. Fine-tuning Qwen3-VL-2B on XVR yields a 1.8× accuracy gain on the internal benchmark and surpasses all closed-source models including GPT-5. XVR-trained VLM backbones further improve robotic manipulation success rates by 13% absolute on RoboCasa.

Problem

Existing spatial reasoning benchmarks and VQA datasets are almost exclusively single-view, limiting spatial information due to occlusions. Recent multi-view datasets focus on what objects appear in each view rather than how viewpoints relate geometrically. No large-scale training dataset provides explicit supervision on cross-view geometric relationships spanning both general and robotic domains, leaving VLMs unable to enforce spatial consistency across viewpoints.

Method

XVR draws on Structure-from-Motion (SfM) primitives to define three reasoning categories operationalized as eight tasks: - Correspondence (Point, Directional): match same 3D location/orientation across views via 3D-to-2D projection from calibrated camera parameters. - Verification (Spatial, Temporal): detect geometrically inconsistent correspondences or temporally misaligned frames; SSIM-based filtering removes trivially distinguishable pairs. - Localization (Viewpoint, Directional View, Cross-Scenario, Language-Conditioned): infer relative camera positions, directional relationships, and language-described viewpoints.

Data comes from two sources: (1) General domain — WildRGB-D calibrated multi-view captures (≥1M point clouds) for geometry-based tasks; (2) Robotic domain — OXE + AgiBot-World trajectories filtered to ≥3 distinct camera views, ≥20s with sufficient end-effector motion. A unified generation framework \(G : (I, P, X, \mathcal{T}, M) \rightarrow (Q, A)\) converts raw multi-view data into multiple-choice QA pairs with spatially separated distractors.

For VLA transfer, a diffusion action head is attached to XVR-fine-tuned Qwen3-VL-2B following GR00T-N1.5 architecture, trained on RoboCasa simulation data.

Key Contributions

  • XVR dataset: 100K multi-view VQA samples from 18K 3D scenes + 70K robotic trajectories, averaging 4.32 images/sample (highest among training datasets).
  • Three-category taxonomy (Correspondence, Verification, Localization) with eight concrete tasks grounded in SfM principles.
  • Comprehensive evaluation across 10 VLMs (open- and closed-source) on XVR-Eval (1,866 held-out samples from unseen sources).
  • Cross-benchmark transfer: improvements on MindCube-Tiny and RoboSpatial-Home despite substantial distribution shift.
  • Embodied transfer: XVR backbone improves VLA manipulation success rates on RoboCasa (+13% absolute average).

Results

  • Qwen3-VL-2B-XVR achieves 68.06% overall on XVR-Eval vs. 36.82% for base Qwen3-VL-2B (1.85× gain) and 61.74% for GPT-5, ranking first among all evaluated models.
  • Point Correspondence: Qwen3-VL-2B-XVR scores 94.32% vs. human baseline 92.31% — super-human performance.
  • Spatial Verification: 84.85%, surpassing GPT-5 (68.56%) and human (88.46%).
  • Most open-source zero-shot models score near chance level (random baseline: 32.64%); Gemini-Robotics-ER-1.5 scores 6.22% on Viewpoint Localization, below random (22.22%).
  • MindCube-Tiny: largest gain on Among subtask (+7.0%).
  • RoboSpatial-Home: largest gain on Compatibility (+7.6%).
  • RoboCasa VLA: +13% absolute average success rate across CoffeePressButton, TurnOffMicrowave, PnPCabToCounter; largest gain on TurnOffMicrowave.
  • Temporal Verification declines after XVR training (base: 45.29% → XVR: 41.18%), the only task showing regression.

Limitations

  • Temporal Verification degrades after XVR fine-tuning due to dataset bias toward spatially synchronized frames; temporal dynamics and orientation reasoning remain weaker than human baselines.
  • XVR consists of 50% static multi-view scenes and 50% robotic trajectories with static camera configurations; tasks involving continuous camera motion (MindCube Around, Rotation) show minimal transfer.
  • XVR training data focuses on inside-looking-outward camera setups; outside-looking-inward configurations (e.g., MindCube) represent an untested distribution.
  • VLA experiments are limited to simulation (RoboCasa); real-world transfer is not evaluated.
  • Directional Correspondence and Temporal Verification still lag human performance by >10 and >35 percentage points, respectively.

Relevance to Vision-Language Models

XVR directly targets a fundamental gap in VLM training: the absence of geometric cross-view supervision that most current benchmarks and datasets do not provide. The finding that a 2B-parameter model fine-tuned on XVR outperforms GPT-5 and Gemini-2.5-Pro on multi-view spatial tasks demonstrates that data quality and explicit geometric supervision dominate model scale for this capability class. The SfM-inspired task taxonomy provides a reusable framework for diagnosing VLM spatial reasoning failures across the Correspondence–Verification–Localization spectrum. Critically, XVR bridges VLMs and embodied AI: the 13% manipulation improvement shows that cross-view relation reasoning is not merely a benchmark artifact but transfers to downstream VLA policy quality, making this dataset directly actionable for robotics-oriented VLM research.