Skip to content

Asking like Socrates: Socrates helps VLMs understand remote sensing images

🕒 Published (v1): 2025-11-27 12:19 UTC · Source: Arxiv · Venue: CVPR 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Existing multimodal reasoning models fail on remote sensing (RS) imagery due to the "Glance Effect"—a single coarse perception pass yields pseudo-reasoning (narrative without genuine visual grounding). RS-EoT addresses this with an iterative reasoning–perception loop instilled via a Socratic multi-agent data synthesis pipeline and a two-stage progressive RL strategy. The resulting RS-EoT-7B achieves state-of-the-art on multiple RS VQA and grounding benchmarks.

Problem

Vision-language models (VLMs) trained with SFT+RL reasoning paradigms (Ă  la DeepSeek-R1) degrade or show no gain on remote sensing tasks compared to their base models. The root cause is the Glance Effect: RS images have wide spatial extents, large scale variation, and sparse cues, so a single-pass global perception leaves the model reasoning from incomplete visual evidence, causing it to fall back on linguistic self-consistency rather than visual facts.

Method

RS-EoT (Remote Sensing Evidence-of-Thought): A language-driven iterative reasoning paradigm where the model alternates between linguistic deduction and targeted visual re-inspection of the RS image, progressively refining its hypothesis at each step.

SocraticAgent (SFT cold-start data synthesis): A self-play multi-agent system with two roles: - Reasoner (GPT-4o-mini): text-only; formulates hypotheses and poses perceptual sub-questions. - Perceiver (Gemini-2.5-flash): vision-only; answers targeted questions about the image.

A self-play prompting mechanism tells each agent that its partner is "weak," forcing the Reasoner to decompose queries into simple incremental questions and the Perceiver to return concise answers. A Verifier (Doubao-seed-1.6-thinking) filters traces where the Reasoner's final answer matches ground truth. This yields RS-EoT-4K (4.3K SFT samples across RGB, infrared, SAR).

Two-stage progressive RL: 1. Stage 1 – RL-Grounding: RL on fine-grained grounding tasks with IoU-based reward, sharpening the visual evidence-seeking behavior. 2. Stage 2 – RL-VQA: RL on general RS VQA, but simple Yes/No datasets are restructured into multiple-choice questions (one image, 10–15 QA pairs, some answers inverted) with a graded symmetric accuracy reward to prevent reward hacking.

Key Contributions

  • RS-EoT paradigm: iterative reasoning–perception loop explicitly tailored to remote sensing imagery characteristics.
  • SocraticAgent: self-play multi-agent trace synthesis (Reasoner + Perceiver + Verifier) with a self-play prompting mechanism to generate structured, progressive evidence-seeking traces.
  • RS-EoT-4K dataset spanning RGB/infrared/SAR modalities.
  • Multiple-choice VQA data reconstruction strategy with a graded reward function enabling stable RL on simple RS VQA corpora.
  • RS-EoT-7B (Qwen2.5-VL-7B backbone): SOTA on RS VQA (RSFG-VQA, VRSBench, RSVQA) and grounding (DIOR-RSVG, VRSBench-Ref).

Results

  • RSFG-VQA Avg@5: RS-EoT-7B 67.85 vs. Qwen2.5-VL 62.45, WeThink 55.04, Geo-R1 45.03.
  • RSVQA Avg@5: 75.16 vs. Qwen2.5-VL 67.20, WeThink 40.74, VHM-RL 67.29.
  • DIOR-RSVG IoU@50: 47.00 vs. Qwen2.5-VL 35.40, VHM-RL 44.63, WeThink 34.51; all other reasoning models below 25.
  • VRSBench-Ref IoU@50: 54.71 vs. Qwen2.5-VL 19.10, VHM-RL 33.13, WeThink 35.56.
  • SocraticAgent vs. frontier distillation (FIT-RSFG-VQA, 2K subset): SocraticAgent Avg@5 64.82 vs. Qwen3-VL-plus 63.78, GLM-4.5V 66.25, Doubao 51.80; SocraticAgent Pass@5 85.89 outperforms all.
  • Ablation: SFT alone improves VQA but collapses grounding (IoU@50 drops from 35.40 to 8.81); RL-IoU restores grounding (48.41); RL-VQA further boosts VQA while maintaining grounding.
  • Attention dynamics show clear periodic peaks on image tokens (evidence-seeking phases) confirming iterative behavior.

Limitations

  • RS-EoT-4K is small (4.3K samples); scaling the synthesis pipeline's impact is unexamined.
  • SocraticAgent relies on proprietary frontier models (GPT-4o-mini as Reasoner, Gemini-2.5-flash as Perceiver, Doubao as Verifier), introducing cost and reproducibility constraints.
  • SFT cold-start severely hurts grounding (IoU@50: 35.40→8.81), requiring RL-IoU recovery; the interaction between SFT and spatial abilities is not fully analyzed.
  • Evaluation limited to standard RS VQA/grounding benchmarks; no real-world agentic deployment or human evaluation.
  • The self-play prompting mechanism's sensitivity to prompt phrasing is not ablated.

Relevance to Agentic AI / LLM Agents

RS-EoT is directly relevant as an instance of agentic loop design: instead of a single-pass LLM call, the model autonomously decides when to re-query its perceptual subsystem, forming a language-directed agent that iteratively gathers evidence before committing to an answer. SocraticAgent is a concrete multi-agent data synthesis pattern—Reasoner/Perceiver/Verifier roles with self-play prompting—that generalizes beyond RS to any domain where a text-reasoning agent must delegate observation tasks to a specialized tool or oracle. The two-stage RL pipeline (capability sharpening then generalization) mirrors hierarchical skill acquisition strategies being explored in broader LLM agent training, and the reward hacking mitigation via multiple-choice reconstruction offers a practical lesson for RL-based agent training on real-world, low-diversity datasets.