Skip to content

RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning

🕒 Published (v1): 2026-06-26 16:57 UTC · Source: Arxiv · Venue: ECCV 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RSICCLLM is the first post-training framework for large vision-language models applied to Remote Sensing Image Change Captioning (RSICC). It introduces a data generation paradigm yielding 40k instruction samples (RSICI) and 20k preference pairs (RSICP), along with Difference-aware Supervised Fine-tuning and Dual-Negative Preference Optimization to explicitly model temporal changes. At only 7B parameters, it outperforms models up to 241B across all metrics.

Problem

Existing RSICC methods rely on conventional deep learning architectures with limited model capacity, constraining performance. Large-model post-training techniques have succeeded in general domains but cannot be directly transferred to RSICC due to data scarcity (no large-scale instruction or preference datasets for change captioning) and the need for fine-grained temporal change understanding that generic VLMs lack.

Method

  • Instruction data generation: Uses Qwen-VL-Max with bi-temporal images and binary change masks as geometric priors to synthesize fine-grained change descriptions, producing the RSICI dataset (~40k samples, ~77 words/caption on average). Integrates LEVIR-CD and SYSU-CD for training (32k), and S2Looking and CDD for out-of-domain evaluation.
  • Difference-aware Supervised Fine-tuning on Qwen2.5-VL-7B: Computes an initial difference map \(T = I_1 - I_2\); extracts time-invariant geometric textures via 8 Central Difference Convolution (CDConv) kernels (standard conv, CDConv, 4 directional asymmetric, 2 symmetric operators); projects each response map through Hough Transform to accumulate geometric evidence; applies inverse Hough independently per timestamp to avoid semantic loss from early differencing; fuses the resulting \(\Delta S\) with \(T\) via cross-attention to produce enhanced visual embeddings \(V_{\mathrm{enhanced}} = V_{\mathrm{orig}} + F\).
  • Dual-Negative Preference Optimization (DNPO): Constructs preference dataset RSICP (20k pairs) via two complementary strategies — (1) info-filtered selection: generates K candidate captions, filters to keywords (object/direction/verb/quantity), selects negatives with intermediate BLEU/ROUGE similarity to ground truth; (2) replacement-based construction: replaces keywords in ground truth with contrastive tokens from the same semantic group. Uses DPO loss \(\mathcal{L}_{\mathrm{DPO}}\) plus a reverse-KL stabilization term \(\mathcal{L}_{\mathrm{S}}\) to keep the trained model close to the reference.

Key Contributions

  • First post-training framework (SFT + preference optimization) for large VLMs in RSICC.
  • Data generation paradigm that produces 40k high-quality instruction samples (RSICI) and 20k preference pairs (RSICP) with automatic quality filtering.
  • Difference-aware SFT with CDConv kernels and Hough Transform for explicit change representation injection into the vision encoder.
  • DNPO with two complementary negative-sample construction strategies and a stabilized DPO objective.

Results

  • On the in-domain RSICI test set, RSICCLLM at 7B achieves: BLEU-4 = 62.78, ROUGE-L = 38.30, SBS = 82.72. The baseline Qwen2.5-VL-7B scores: BLEU-4 = 0.61, ROUGE-L = 10.16, SBS = 42.06 — showing a massive improvement from domain-specific post-training.
  • Outperforms all larger models including InternVL-3.5-241B, Qwen3-VL-235B, Qwen2.5-VL-72B, and proprietary models (Qwen-VL-Max, GLM-4.5V, SenseChat-Vision).
  • On the out-of-domain test set: RSICCLLM achieves BLEU-4 = 47.84, ROUGE-L = 25.89, SBS = 73.86, substantially ahead of Qwen2.5-VL-72B (BLEU-4 = 3.06, ROUGE-L = 20.25, SBS = 68.96).
  • Domain-specific RSICC methods (TEOChat-7B, CCExpert-7B) perform poorly with BLEU-4 < 1.1 and SBS < 42, underscoring the gap RSICCLLM fills.

Limitations

  • Data generation depends on Qwen-VL-Max as the teacher model, inheriting its biases and quality ceiling.
  • Evaluated primarily on n-gram overlap metrics (BLEU, ROUGE); SBS partially addresses semantics but no human evaluation is reported.
  • Limited to 7B parameter scale; performance scaling to larger backbones is unexplored.
  • Preference dataset construction relies on heuristics (keyword extraction, similarity thresholds \(\tau_1, \tau_2\)) that may not generalize across domains.
  • CDConv and Hough Transform modules introduce task-specific architectural modifications that may not transfer to other captioning tasks.

Relevance to Vision-Language Models

This paper demonstrates that domain-specific post-training — combining supervised fine-tuning with explicit architectural inductive biases (difference maps, geometric texture features) and preference optimization — can dramatically improve VLM performance on specialized tasks where generic large VLMs fail. It provides a template for adapting general-purpose VLMs to fine-grained temporal reasoning in vision, which is relevant beyond remote sensing to any change-understanding task. The data generation pipeline using a strong VLM as teacher also illustrates a practical path for creating high-quality instruction data in data-scarce domains.