Skip to content

GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

GRE Suite introduces a reasoning-augmented framework for worldwide image geo-localization built around structured chain-of-thought (CoT) reasoning. It combines a curated 30K-sample dataset (GRE30K), a three-stage post-training pipeline (SFT cold-start + two-stage GRPO reinforcement learning), and a new evaluation benchmark (GREval-Bench) that assesses both localization accuracy and reasoning quality. The resulting GRE model (Qwen2.5-VL-7B backbone) surpasses all prior SOTA methods on Im2GPS3k and GWS15k while training on only 5% of the MP-16 dataset.

Problem

Existing geo-localization methods rely on data-driven cross-modal alignment (retrieval, classification grids, or RAG over GPS galleries), which ignores logical relationships among fine-grained geographic indicators and requires large-scale databases. They also produce closed-domain, discrete coordinate predictions rather than open-ended inference. Additionally, current benchmarks evaluate only terminal prediction accuracy and ignore the intermediate reasoning quality, limiting interpretability and model improvement.

Method

GRE follows a three-stage post-training pipeline on Qwen2.5-VL-7B:

  1. GRE30K construction: GPT-o3 generates image–CoT–coordinate triples from MP-16-Pro images. After automated filtering and manual verification, this yields GRE30K-CoT (20K high-quality CoT samples for SFT) and GRE30K-Judge (10K judgment tasks where predictions are labeled True/False against ground truth using a geodesic threshold θ, used for RL Stage I).

  2. Cold-start SFT (GRE-CI): Fine-tunes the base model on GRE30K-CoT to establish structured <think>…</think><answer>…</answer> output format and systematic reasoning patterns.

  3. Two-stage GRPO RL:

  4. Stage I: Uses GRE30K-Judge; the model evaluates correctness of a provided reasoning chain + answer (binary accuracy reward: 1 if prediction matches ground truth label, else 0; plus format reward enforcing think/answer tags).
  5. Stage II: Uses 170K raw seed images; model directly predicts coordinates with a continuous geodesic reward R_geo = 2/(1 + exp(d/θ)) based on great-circle distance.

GRPO clips policy update ratios to [1−ε, 1+ε] and adds KL regularization against a reference policy for training stability.

GREval-Bench is a new 3K-triplet benchmark containing filtered geo-informative images (2400 outdoor, 600 indoor), per-image corpora of explicit and implicit geographic indicators, and annotated CoT steps. CoT quality is measured via a composite score: (Recall vs. geography corpus + RefCLIPScore for image captions + BERTScore for logical inference steps) / 3.

Key Contributions

  • GRE30K: 30K-sample geo-localization reasoning dataset (20K CoT + 10K judgment tasks) generated via GPT-o3 with manual filtering.
  • GRE model: Three-stage post-training (SFT cold-start + two-stage GRPO RL) on Qwen2.5-VL-7B enabling open-ended coordinate prediction via structured geographic reasoning.
  • GREval-Bench: 3K-triplet evaluation benchmark with explicit/implicit indicator corpora and CoT quality scoring, addressing gaps in existing benchmarks (Im2GPS3k, GWS15k) that lack reasoning assessment.
  • Novel two-stage reward design: binary judgment reward (Stage I) followed by continuous geodesic reward (Stage II).

Results

Im2GPS3k (vs. prior SOTA GeoCLIP / GeoReasoner): - GRE achieves 11.3 / 35.3 / 51.7 / 69.3 / 85.7% at 1/25/200/750/2500 km thresholds. - Improvements over GeoCLIP: +0.5 / +4.2 / +3.0 / +1.7 / +2.5 pp across all thresholds. - Trained on only 5% of MP-16 vs. full dataset used by baselines.

GWS15k (geographically uniform, large distribution shift): - GRE achieves 0.9 / 4.1 / 18.9 / 54.8 / 78.3% at 1/25/200/750/2500 km. - Improvements over GeoCLIP: +0.3 / +1.0 / +2.0 / +9.1 / +4.2 pp.

GREval-Bench: - GRE-7B: 6.14 / 26.15 / 44.67 / 66.56 / 83.16% (localization) + CoT quality 59.54. - Outperforms Qwen2.5VL-32B (5.45 / 23.12 / 37.41 / 54.33 / 65.00%, CoT 55.56) despite 4× fewer parameters.

Ablation: CI alone gives large gains; Stage II RL drives most of the fine-grained improvement; combined CI+I+II is best overall.

Limitations

  • Base model is limited to Qwen2.5-VL-7B; larger-scale RL training experiments are in the appendix but not fully reported in the main paper.
  • GRE30K is constructed from MP-16 (Flickr images), which has geographic and content biases; the filtering reduces but may not eliminate these.
  • Stage I RL reward (binary judgment) creates a task-type mismatch with the test setting (direct coordinate prediction), causing some metric drops at certain granularities when used alone.
  • GREval-Bench is semi-automated (GPT-4o for step categorization); annotation quality inherits GPT-4o's errors.
  • Open-ended coordinate prediction without a candidate database may degrade on very fine-grained (street-level, 1 km) localization where retrieval-based methods have complementary strengths.

Relevance to Vision-Language Models

This work directly demonstrates how RL-based reasoning enhancement (GRPO, inspired by DeepSeek-R1) can be transferred from language tasks to a spatially grounded, multi-step VLM task—showing that structured CoT reasoning is a general mechanism, not limited to math or code. The cold-start SFT → two-stage RL recipe is a reusable post-training pattern for any VLM task requiring progressive multi-granular inference. The GREval-Bench's composite CoT quality metric is a novel evaluation contribution for the broader VLM reasoning evaluation community. The finding that a 7B reasoning-augmented model outperforms a 32B generalist on both localization accuracy and CoT quality has direct implications for efficient VLM specialization.