Skip to content

SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion

🕒 Published (v1): 2026-06-26 09:16 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SpatialUAV is a real low-altitude UAV benchmark with 4,331 instances across 14 task types covering semantic discrimination, spatial relation, aerial–aerial/aerial–ground collaboration, and motion understanding. Evaluations of 18 VLMs show a large gap from human-level performance (89.0% vs. 56.7% best model), with severe bottlenecks in cross-view association, geometric reasoning, and temporal viewpoint understanding.

Problem

Existing spatial intelligence benchmarks remain grounded in human-centered, indoor, or ground-level perspectives (e.g., SpatialVLM, VSI-Bench, SQA3D) and do not capture low-altitude UAV challenges: perspective distortion, altitude-dependent scale variation, occlusion, and aerial–ground viewpoint mismatch. Current UAV benchmarks offer incomplete spatial reasoning (mostly image-level recognition), limited multi-view collaboration evaluation, underexplored scene dynamics and ego-motion, and overly narrow input/output formats.

Method

SpatialUAV organizes each instance as a tuple (visual input \(\mathcal{V}\), task type \(\tau\), question \(q\), answer \(y^*\)). Visual inputs cover \(K \in \{1, 2, 5, 16\}\) frames for single images, paired views, candidate selection, and video. The construction pipeline has five stages:

  1. Data Collection — five data sources: BEDI, AirCopBench, MAVREC, AirScape, University-1652.
  2. Task Synthesis — per-task construction of visual inputs (detected/clean/annotated images), questions (template-based, fixed, or manually written), and answers (rule-based from metadata/depth, or manual annotation).
  3. Standardization — all samples serialized into a unified record schema with nine answer formats: option letters, region identifiers, region-pair lists, bounding boxes, clock directions, angle values, angle–distance pairs, direction labels, and free-form text.
  4. Blind Filtering — DeepSeek-V4-Pro and Qwen3.6-27B receive text-only prompts; correctly answered samples are removed to eliminate language priors.
  5. Hybrid Validation — exhaustive human cross-validation followed by model-assisted review (Qwen3-VL-30B, Qwen3.6-35B, InternVL3.5-38B); samples where all three models agree on a wrong answer are rechecked.

The final benchmark has 4,331 instances: 1,315 single-image, 1,231 aerial–aerial, 785 aerial–ground, and 1,000 video-motion.

Key Contributions

  • SpatialUAV benchmark: 4,331 curated real low-altitude UAV instances with 14 fine-grained task types spanning semantic discrimination, spatial relation, aerial–aerial collaboration, aerial–ground collaboration, and motion understanding.
  • Unified multi-format pipeline: supports 7 input configurations and 9 answer formats with multi-source supervision (detector regions, depth maps, metadata rules, manual labels) and task-specific metrics.
  • Diagnostic evaluation of 18 VLMs across closed-source, spatial-specific, and open-source categories, revealing critical capability gaps and providing empirical guidance for future low-altitude UAV spatial intelligence research.

Results

  • Human-level macro-average: 89.0%.
  • Random choice baseline: 14.4%.
  • Closed-source models: GPT-5.4 (56.7%), Claude-Opus-4-7 (51.3%), Gemini-3.1-Flash (45.9%).
  • Best open-source model: Qwen3.6-27B (49.8%), followed by Qwen3.6-35B-A3B (37.3%) and Qwen3.5-9B (42.5%).
  • Spatial-specific models perform worst: VST-7B-SFT (29.7%), SpaceEra++ (28.2%), SpaceEra (27.3%), SpatialVLM (25.7%), Cambrian-S-7B (24.1%).
  • Hardest tasks for top models: Camera Transformation (GPT-5.4: 8.2%, Qwen3.6-27B: 7.4%), Object Matching (GPT-5.4: 28.8%, Qwen3.6-27B: 9.0%), Shared Association (GPT-5.4: 32.8%, Qwen3.6-27B: 24.7%).
  • All models show severe degradation on geometric reasoning (angle/distance estimation) and cross-view correspondence tasks.
  • Top models still perform significantly below human on most groups: multiple-image reasoning is the weakest area for almost all models.

Limitations

  • Moderate scale: 4,331 instances is relatively small compared to large-scale benchmarks like UAVBench (50K).
  • Data source dependency: relies on five existing UAV datasets — coverage may not generalize across all real-world UAV scenarios.
  • Manual annotation cost: several tasks (anomaly localization, cross-view correspondences, occlusion recovery, free-form motion) require extensive manual labeling, limiting scalability.
  • Perception-only evaluation: benchmark measures spatial reasoning from static observations but does not evaluate closed-loop planning or real-time navigation under physical constraints.
  • Free-form evaluation: Global Motion uses free-form text answers, which is harder to evaluate reliably and may introduce metric noise.

Relevance to Vision-Language Models

This benchmark directly targets the gap between current VLM spatial capabilities and the demands of real-world UAV deployment. It reveals that even the strongest VLMs (GPT-5.4, Qwen3.6-27B) fail dramatically at cross-view association, geometric grounding, and ego-motion understanding — capabilities essential for embodied aerial perception. For the VLM research community, SpatialUAV provides a structured diagnostic to isolate which spatial reasoning dimensions require architectural or training-data innovations beyond current large-scale pretraining.