Skip to content

WeaveEarth: Structured Evidence Construction and Reasoning for Training-Free UHR Remote Sensing Understanding

🕒 Published (v1): 2026-07-11 04:56 UTC · Source: Arxiv · link

Why this paper was selected

Training-free UHR remote sensing VLM reasoning with structured evidence; novel efficiency angle

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

WeaveEarth is a training-free framework for Ultra-High-Resolution (UHR) remote sensing image understanding that reformulates the problem as structured evidence construction and reasoning rather than expanded visual access. It constructs a compact Minimal Support Evidence Set (MSES) from overlapping local crops under global context constraints, then organizes selected patches with spatial metadata and topology-preserving layout into a unified interface for a frozen VLM. It achieves state-of-the-art results on multiple UHR benchmarks without any parameter fine-tuning.

Problem

VLMs struggle with UHR remote sensing images (e.g., \(10{,}000 \times 10{,}000\) px) due to a Macro-Micro Duality: global scene layout and sparse yet task-critical local details must be jointly reasoned under limited token budgets. Passive perception methods (resolution scaling, token compression) discard fine-grained evidence; active perception methods (iterative zoom-and-search) incur high latency, contextual fragmentation, error accumulation, and often require additional training for search/stopping policies.

Method

WeaveEarth operates in two stages, both plug-and-play over a frozen VLM backbone:

Stage 1 — Global-Aware Evidence Construction. The UHR image is divided into \(N\) overlapping local patches forming a candidate pool, alongside a low-resolution global thumbnail \(g\). Each patch \(p_i\) is scored jointly against question \(q\) and global thumbnail:

\[s_i = \text{sim}(p_i, q) + \lambda\,\text{sim}(p_i, g)\]

Top-ranked anchors are expanded with spatial neighbors to prevent boundary fragmentation. From this expanded pool \(\mathcal{C}\), a Minimal Support Evidence Set (MSES) \(\mathcal{S}^*\) is constructed via a greedy training-free strategy optimizing:

\[\mathcal{S}^* = \arg\max_{\mathcal{S} \subseteq \mathcal{C}}\; \text{Rel}(\mathcal{S}, q) + \alpha\,\text{Cov}(\mathcal{S}) - \beta\,\text{Red}(\mathcal{S})\]

balancing semantic relevance, spatial coverage/complementarity, and low visual redundancy.

Stage 2 — Structured Evidence Reasoning. Each MSES patch \(p_i\) is encoded as a structured evidence unit \(e_i = (p_i, u_i, b_i, n_i, r_i, \ell_i)\) comprising: local image, region ID, normalized bounding box, adjacency neighbors, role (anchor/support), and grid/scale. These Structured Evidence Metadata (SEM) items are arranged into a Topology-Preserving Evidence Board (TPEB), which lays patches out to preserve coarse-grained spatial adjacency from the original image. The frozen VLM receives three inputs: global thumbnail, TPEB, and SEM text.

Key Contributions

  • Training-free UHR RS understanding framework (WeaveEarth) compatible with any frozen VLM backbone as a plug-and-play inference wrapper
  • Global Context Constraint (GCC): joint question–thumbnail scoring for evidence candidate retrieval, replacing question-only patch retrieval
  • MSES construction via greedy joint optimization of relevance, coverage, and redundancy
  • Structured Evidence Metadata (SEM): explicit per-patch spatial grounding (bounding box, adjacency, role, grid scale)
  • Topology-Preserving Evidence Board (TPEB): visual layout that preserves relative spatial topology among selected patches

Results

  • LRS-VQA (7,333 QA pairs): WeaveEarth (Qwen3-VL-8B backbone) achieves 33.38% avg accuracy, surpassing:
  • Luo et al. (2025): 27.45% (+5.93 pp)
  • ZoomSearch: 30.99% (+2.39 pp)
  • ZoomEarth: 24.56% (+8.82 pp)
  • GeoLLaVA-8K: 25.65% (+7.73 pp)
  • MME-RealWorld (3,738 MCQ): WeaveEarth achieves 47.38% avg, surpassing:
  • ZoomEarth: 43.93% (+3.45 pp)
  • ZoomSearch: 43.02% (+4.36 pp)
  • Best open-source general VLM (Qwen3-VL-8B standalone): 42.11%
  • Results are consistent across three frozen VLM backbones: Qwen3-VL-8B, LLaVA-v1.6-7B, IXC-2.5-7B

Limitations

  • Evaluation covers only remote sensing VQA benchmarks; generalization to other UHR domains (e.g., medical imaging, satellite video) is not demonstrated
  • Performance depends on the quality of the cross-modal semantic encoder used for evidence scoring; encoder choice is not systematically ablated in the presented text
  • The greedy MSES construction is heuristic; optimality of the selected evidence set is not guaranteed
  • Token budget and hyperparameters (\(\lambda\), \(\alpha\), \(\beta\), \(k\)) require tuning per backbone/dataset; sensitivity analysis is deferred to the appendix (not shown in excerpted text)
  • Latency comparison (231.26 s cited for a competitor) is shown qualitatively in Figure 1 but a systematic efficiency table is not reproduced in the presented excerpt

Relevance to Vision-Language Models

WeaveEarth directly addresses a fundamental bottleneck for VLMs operating on images that far exceed standard input resolutions: rather than modifying the model, it redesigns the inference input interface through structured evidence composition. The dual mechanism of SEM (textual spatial grounding) and TPEB (visual spatial layout) provides a spatial inductive bias that compensates for a frozen VLM's inability to reason globally over fragmented local crops — a transferable design principle for any multi-image or high-resolution VLM pipeline. The training-free, backbone-agnostic nature makes it directly applicable to frontier VLMs (Qwen3-VL, LLaVA series) without the data and compute overhead of fine-tuning. For researchers tracking VLMs, this work exemplifies the emerging paradigm of inference-time evidence structuring as an alternative to architectural scaling for resolution extension.