Skip to content

GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

GeoRanker is a distance-aware candidate re-ranking framework for worldwide image geolocalization that uses a large VLM (Qwen2-VL-7B) to jointly encode query–candidate interactions and predict geographic proximity via a scalar distance score. It introduces a multi-order distance loss combining first-order (absolute distance ranking via partial Plackett-Luce) and second-order (relative pairwise distance-gap ranking) objectives. On IM2GPS3K and YFCC4K it sets new state-of-the-art across all geographic granularities.

Problem

Existing two-stage geolocalization pipelines retrieve candidates via embedding similarity but select the final prediction using naïve cosine similarity or point-wise supervision. These methods encode query and candidates independently, failing to model mutual spatial interactions or the relative spatial structure among candidates (e.g., Tobler's First Law). Despite the correct answer often existing in the top-k retrieved set, the selection stage is the bottleneck.

Method

GeoRanker operates as a re-ranking module on top of any retrieval stage. Given a query image and a candidate set (retrieved + LVLM-generated), a structured multimodal prompt is built: {query image} How far is this place from {lat, lon, textual description, candidate image}? Negative examples: {negative info}. This prompt is fed into Qwen2-VL-7B fine-tuned with LoRA (rank 16, targeting q/k/v projections). The final-position hidden state is projected by a single linear value head to a scalar distance score per candidate.

Training uses the GeoRanking dataset (100K samples, 2M query–candidate pairs built from MP16-Pro). The loss is a weighted combination of: - First-order loss (λ=0.7): partial Plackett-Luce loss over candidates sorted by geodesic distance to query (K⁽¹⁾=1 top element). - Second-order loss (1−λ=0.3): partial Plackett-Luce loss over all pairwise distance differences Δd_{i,j}, encouraging larger score gaps for geographically more-separated pairs.

At inference, GeoRanker scores both retrieved candidates (|C_r|=12–14) and LVLM-generated candidates (|C_g|=3–5, GPS-only), selecting the top-scored as the prediction.

Key Contributions

  • GeoRanker: first distance-aware ranking framework for geolocalization using a VLM backbone with LoRA, replacing independent similarity scoring with joint query–candidate spatial reasoning.
  • Multi-order distance loss: novel combination of first-order (absolute distance) and second-order (relative distance-gap) Plackett-Luce objectives for structured spatial supervision.
  • GeoRanking dataset: first ranking dataset for geographic entities; 100K samples with multimodal annotations (GPS, text, images), released publicly.
  • State-of-the-art on IM2GPS3K and YFCC4K across all five geographic thresholds (1km–2500km).

Results

  • IM2GPS3K street-level (1km): 18.79% vs. G3's 16.65% → +12.9% relative gain; prior best non-RAG method GeoCLIP: 14.11%.
  • YFCC4K street-level (1km): 32.94% vs. G3's 23.99% → +37.3% relative gain.
  • IM2GPS3K region (200km): 61.49% (+10.7% over G3 55.56%); country (750km): 76.31% (+5.4%); continent (2500km): 89.29% (+4.7%).
  • YFCC4K city (25km): 43.54% (+21.3%); region (200km): 54.32% (+15.6%).
  • Ablation: removing second-order loss drops continent-level accuracy most (89.29→88.28); removing candidate images is the single most damaging ablation (15.58% street-level).
  • Surpasses G3 even when fine-tuned on only 10% of GeoRanking data (10K samples).
  • Inference latency < half that of prompting-based LVLM selection for 1–10 candidates (parallel scoring vs. sequential prompt construction).
  • Backbone scaling: performance monotonically improves from 0.5B (LLaVA-OneVision) → 2B → 7B (Qwen2-VL).

Limitations

  • Generated candidates (from GPT-4V) lack image/text modalities, reducing their informativeness compared to retrieved candidates at inference.
  • Relies on GPT-4V for candidate generation at inference, introducing a proprietary, cost-bearing dependency.
  • GeoRanking is derived from MP16-Pro; performance on regions underrepresented in that database may be weaker (geographic bias not explicitly analyzed).
  • K⁽¹⁾=1 and λ=0.7 were tuned on IM2GPS3K; transfer to very different distributions is not studied.
  • No analysis of failure modes for visually ambiguous or deceptive scenes (e.g., regions with near-identical architecture across continents).

Relevance to Vision-Language Models

GeoRanker directly demonstrates a high-value application of VLMs as spatial reasoners: rather than using a VLM to generate free-form text, it repurposes the model's cross-modal hidden states as a regression head for geographic distance, fine-tuned via a novel list-wise loss. This is a clean example of adapting pre-trained VLMs (Qwen2-VL) with LoRA for structured prediction tasks beyond captioning or VQA. The work also highlights how VLM scale correlates with downstream task accuracy in a non-generative setting, contributing to the growing evidence of VLM scaling laws for specialized grounding tasks. For researchers studying VLM spatial/geographic understanding, the GeoRanking dataset and the multi-order ranking objective are immediately reusable.