AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration

🕒 Published (v1): 2026-06-26 12:51 UTC · Source: Arxiv · link

Why this paper was selected

ECCV; spatial reasoning benchmark for MLLMs across heterogeneous multi-view embodied settings

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AirGroundBench is a diagnostic benchmark for evaluating geometric-consistency-centered spatial intelligence in MLLMs under heterogeneous UAV–UGV (air–ground) collaboration. It contains 62k dual-view VQA instances and 115 closed-loop VLN episodes across 10 task types spanning four capability dimensions. Evaluation of 13 state-of-the-art MLLMs reveals a consistent capability ladder: perception is handled reasonably well, but cross-view alignment and spatial transformation remain major bottlenecks, with all models far below human performance.

Problem

Existing spatial reasoning benchmarks focus on single-agent, single-view perception or homogeneous multi-camera rigs, leaving heterogeneous multi-view collaboration systematically underexplored. Aerial and ground viewpoints are complementary but introduce scale mismatch, asymmetric occlusion, and reference-frame inconsistency—challenges that current benchmarks cannot diagnose. There is no existing benchmark that jointly provides (i) heterogeneous UAV–UGV dual-view observations, (ii) metric 3D geometry-verifiable annotations with cross-view object identity, and (iii) both static VQA and closed-loop VLN evaluation.

Method

AirGroundBench is constructed in Unreal Engine + AirSim simulation across 11 diverse environments (urban, coastal, desert, mountain, etc.). Synchronized UAV–UGV image pairs are captured by composing the UAV pose in the UGV-local NED frame:

\[\mathbf{T}^{w}_{u}(t) = \mathbf{T}^{w}_{g}(t)\,\mathbf{T}^{g}_{u}\]

where \(\mathbf{T}^{g}_{u} \in SE(3)\) is a preset-defined relative transformation, ensuring reproducible cross-view geometry. Annotations include cross-view object identities and metric 2D/3D bounding boxes. Questions are generated via a semi-automatic pipeline: template-driven generation from structured annotations → model-assisted linguistic diversification → human verification. Ten tasks are organized into four capability dimensions: spatial perception (P-Sce, P-Obj, P-Bear), cross-view alignment (A-Scale, A-Match, A-Dist), spatial transformation and reasoning (R-CF, R-View, R-Loc), and embodied decision-making (VLN). All VQA tasks are 4-way single-choice; VLN episodes are evaluated via Success Rate (SR), SPL, and Navigation Error (NE) in a closed-loop discrete-action environment. Models are evaluated zero-shot under three input conditions: UGV-only, UAV-only, and dual-view.

Key Contributions

• A geometry-consistency-centered benchmark uniquely combining heterogeneous UAV–UGV dual-view observations with metric 3D ground-truth annotations and both static VQA and interactive VLN evaluation.
• 1,021 synchronized UAV–UGV image pairs across 11 environments, yielding 62k single-choice VQA questions and 115 VLN episodes, with auxiliary depth/segmentation and simulator state releases.
• A four-dimension task taxonomy (perception → alignment → transformation/reasoning → decision-making) that localizes where geometric consistency breaks down.
• Large-scale zero-shot evaluation of 13 MLLMs (proprietary and open-source) with diagnostic ablations under single-view vs. dual-view inputs.

Results

• Best proprietary model (Doubao-Seed-2.0-Pro): 57.14% average VQA accuracy vs. ~80% human; best open-source (Qwen3.5-397B): 55.11%.
• Random baseline: ~25% (consistent with 4-way chance).
• Perception tasks are comparatively strongest; alignment (A-Scale, A-Match, A-Dist) and reasoning (R-View, R-Loc) show the sharpest drops across all models.
• VLN: best SR/SPL is 38.71%/31.45% (Doubao) vs. human 83.87%/68.37%; navigation error remains high for all models.
• Dual-view inputs provide measurable gains over single-view variants, but the gap to human performance persists across all 13 models.
• Hunyuan-Vision scores lowest overall (34.42% avg.), confirming the benchmark is discriminative across capability tiers.
• Performance is consistent across City vs. Wild environment splits, indicating challenges are not environment-specific.

Limitations

• Benchmark is simulation-only (Unreal Engine/AirSim); real-world domain gap is unaddressed.
• Only 115 VLN episodes, which is relatively small for reliable sequential evaluation.
• Discrete action space in VLN may not reflect continuous robot control challenges.
• UAV viewpoints are constrained to preset relative poses rather than free-flight trajectories, limiting viewpoint diversity.
• Paper is truncated; full ablation results on single-view vs. dual-view contribution and environment-level breakdowns are partially cut off.

Relevance to Vision-Language Models

AirGroundBench directly probes a known weak point of VLMs—geometric consistency across heterogeneous viewpoints—in a setting (embodied UAV–UGV collaboration) that prior benchmarks have not covered. The finding that all 13 evaluated MLLMs plateau in the mid-50s on cross-view alignment and transformation tasks, despite strong general vision-language competence, quantifies a concrete capability gap relevant to VLM development. The closed-loop VLN component is particularly valuable for the VLM community because it demonstrates how one-step spatial reasoning failures compound into sequential navigation errors—a regime where descriptive spatial understanding does not imply action-consistent understanding. This benchmark provides a structured testbed for future work on multi-view geometric reasoning, spatial grounding, and embodied VLM evaluation.