Skip to content

InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

InternVL team (Zhe Chen); ICLR 2026; large-scale spatial reasoning dataset for VLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

InternSpatial is the largest open-source spatial reasoning dataset for VLMs, containing 12M QA pairs spanning single- and multi-view scenarios across 19 instruction formats. It is accompanied by InternSpatial-Bench (6,008 QA pairs), a diagnostic benchmark including a novel rotation estimation task. Fine-tuning InternVL2.5-8B on InternSpatial yields +12.1% on InternSpatial-Bench and +10.7% on VSI-Bench over the base model.

Problem

Existing spatial reasoning datasets for VLMs suffer from three compounding limitations: (1) narrow scene diversity (predominantly indoor or outdoor only), (2) single instruction format per dataset (natural language only, or region masks only), and (3) restriction to single-view settings, leaving multi-view spatial reasoning without adequate training supervision. SpatialVLM's 2B-pair dataset, the largest prior work, remains closed-source.

Method

InternSpatial is constructed through a fully automated, modular data engine operating in three stages:

  1. Image-level annotation generation: For datasets lacking annotations, open-source VLMs extract 2D bounding boxes and captions; SAM2 produces segmentation masks. For 3D datasets, 3D bounding boxes are projected onto image planes. Filtering removes occluded, boundary-clipped, or categorically ambiguous objects (e.g., sky, grass).

  2. View-space alignment: All annotations are lifted to a canonical view space centered at the camera's optical center (y-axis = viewing direction, z-axis = upward). For image-only sources without camera parameters, the pipeline uses WildCamera (intrinsics), PerspectiveFields (extrinsics), and Metric3Dv2 (dense depth) to estimate 3D structure, following SpatialRGPT.

  3. Template-based QA generation: An LLM pre-generates templates with placeholders for each task type. At scale, templates are instantiated by filling in object references and programmatically derived answers—no per-image LLM calls. Each base QA is expanded into up to 19 variants by combining four image formats (original, bounding-box overlay, segmentation mask overlay, numbered-region overlay) with five textual reference styles (natural language, <ref>, <box>, <ref><box>, image-grounded text).

For multi-view data, scene geometry is derived from ScanNet, MultiScan, R2R, and Objaverse point clouds; room dimensions are estimated via the Alpha Shape algorithm; object annotations use OrientedBoundingBox format from Open3D. A novel rotation estimation task introduces 2.46M cross-view QA pairs. A 500-item manual validation confirms >95% QA accuracy.

InternSpatial-Bench (6,008 QA pairs) extends and refines SpatialRGPT-Bench and SpatialBench by diversifying instruction formats, adding curated QA pairs, and introducing the rotation estimation task. All benchmark entries are manually verified.

Key Contributions

  • InternSpatial: 12M open-source QA pairs (9.5M single-view + 2.5M multi-view) covering 5 scene domains (in-the-wild, indoor, street, object-centric, embodied navigation) and 19 instruction formats—the largest open-source spatial reasoning dataset for VLMs.
  • InternSpatial-Bench: 6,008-pair benchmark for 5 tasks (Position Estimation, Size Estimation, Rotation Estimation, Existence Estimation, Object Counting) spanning diverse domains and instruction formats, including the first rotation estimation evaluation task.
  • Novel rotation estimation task: 2.46M training pairs and a corresponding benchmark task targeting cross-view viewpoint change reasoning, a gap absent from all prior datasets.
  • Modular, LLM-free data engine enabling scalable, heterogeneous-source QA generation with 19-format expansion without per-image LLM inference cost.

Results

  • InternSpatial-Bench: InternVL-Spatial-8B achieves +12.1% over InternVL2.5-8B baseline.
  • VSI-Bench (multi-view spatial reasoning): InternVL-Spatial-8B achieves +10.7% over InternVL2.5-8B baseline.
  • Gains transfer across architectures: improvements also demonstrated on InternVL2.5-1B (InternVL-Spatial-1B) and Qwen2.5-VL-8B (Qwen-Spatial-8B).
  • General multimodal performance on standard VL benchmarks is preserved (no regression), confirmed via ablation across general-purpose tasks.
  • Human validation on 500 sampled QA pairs confirms >95% accuracy in the generated training data.

Limitations

  • Template-based QA generation, while scalable, produces structurally repetitive question patterns compared to LLM-generated diversity; the paper mitigates but does not eliminate this.
  • Manual verification covers only a 500-item random sample out of 12M pairs; systematic biases in automated annotations (depth estimation errors, projection artifacts) cannot be fully audited.
  • Rotation estimation task is currently object-centric (Objaverse, R2R); generalization to complex multi-object multi-view scenes is not directly evaluated.
  • Experiments are limited to InternVL2.5-8B, InternVL2.5-1B, and Qwen2.5-VL-8B; it is unclear how well gains transfer to proprietary or larger-scale models.
  • InternSpatial-Bench's rotation estimation subset uses only original image format + natural language (no format diversity), which may underestimate difficulty in format-varied settings.

Relevance to Vision-Language Models

Spatial reasoning is a persistent blind spot for VLMs despite strong performance on standard VQA and captioning tasks; InternSpatial directly attacks the data scarcity bottleneck by providing the first large-scale, open-source, multi-format, multi-view training corpus at this scale. The 19-instruction-format design is particularly significant for VLM research: it systematically bridges the gap between models that process text-only queries and those that handle visual grounding signals (bounding boxes, masks, numbered overlays), enabling more robust instruction-following across spatial tasks. The rotation estimation task introduces a novel evaluation axis not present in prior benchmarks, relevant to embodied AI, robotics, and scene understanding. By demonstrating gains on VSI-Bench without general regression, InternSpatial establishes a viable SFT recipe for spatial specialization that complements rather than degrades a VLM's general capabilities.