SD-MAR: Multi-image Analytical Reasoning via Synthetic Data and Reinforcement Learning

🕒 Published (v1): 2026-07-15 19:55 UTC · Source: Arxiv · link

Why this paper was selected

RL + synthetic data for multi-image analytical reasoning; addresses core VLM reasoning gap

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SD-MAR is a framework that combines a synthetic data generation pipeline with a modified reinforcement learning algorithm (GRPO-lite + BDA) to train VLMs on multi-image analytical reasoning—tasks requiring cross-image comparison, change detection, and multi-step inference grounded in visual differences. Fine-tuning Qwen2.5-VL-7B yields up to 36.95% in-domain accuracy gains and surpasses GPT-4.1 on the SD-MAR benchmark, while general-purpose multimodal performance is preserved within ±1% on MME, MMMU-Pro, and MathVista.

Problem

Current VLMs are strong at single-image perception but fail at tasks requiring systematic reasoning across multiple visual states (comparison, temporal change detection, causal attribution). Existing benchmarks do not jointly evaluate controlled visual transformation, multi-image comparison, and multi-step analytical inference, leaving this capability under-studied and under-optimized.

Method

SD-MAR SDG constructs two task families: - SD-MAR-TSE: Paired images generated via hierarchical attribute perturbations (temporal, spatial, emotional). Captions are produced by Claude Sonnet 3.7; images are rendered by Amazon Nova-Pro; quality is filtered via CLIPScore thresholding. Questions require attributing the root cause of the change (multi-choice). - SD-MAR-Math: Real-world World Bank GDP time series rendered as bar charts, line charts, or tables across disjoint intervals. Three stratified difficulty levels: L1 (absolute difference), L2 (percentage change between interval averages), L3 (multi-country growth-rate leader identification).

SD-MAR RLVR uses GRPO-lite, a targeted modification of GRPO with three changes: 1. KL regularization removed: allows the policy to develop cross-image comparison strategies not present in the pretrained distribution, with clipping as the sole stability constraint. 2. Advantage normalization removed: preserves the natural reward skew across SD-MAR's stratified difficulty levels, avoiding gradient instability caused by conflating easy (L1) and hard (L3) instances. 3. Backward Discounted Allocation (BDA): assigns token-level rewards $\(r_t = R \cdot \frac{\gamma^{L-1-t}}{\sum_{i=0}^{L-1} \gamma^i}, \quad \gamma \in (0,1)\)$ placing higher credit on later reasoning tokens (comparison/inference steps) than early description tokens, reflecting the perceive→compare→infer structure of MAR.

Outcome reward is binary accuracy (\(w_{acc}=0.7\)) plus format validity (\(w_{fmt}=0.3\)); training uses VeRL.

Key Contributions

• Formal definition of multi-image analytical reasoning (MAR) as inference over pairwise visual differences \(a = f(\{\Delta_{ij}\}, q)\).
• SD-MAR synthetic data generation pipeline (TSE + Math, ~21k examples total) with verifiable ground-truth answers enabling RLVR.
• GRPO-lite with BDA: a principled, MAR-specific RL configuration that removes KL regularization and advantage normalization and introduces backward-discounted credit assignment.
• Demonstration that targeted synthetic RLVR fine-tuning improves a specific multimodal capability without catastrophic forgetting on unrelated benchmarks.

Results

• In-domain (SD-MAR-Mix):
• Qwen2.5-VL-7B: +29.0% average accuracy; Math subset 48.8% → 91.73%
• InternVL3-8B: +15.4% average accuracy; Math subset 40.53% → 60.40%
• Up to 36.95% overall in-domain improvement; Qwen2.5-VL-7B surpasses GPT-4.1 on the SD-MAR test set
• BDA ablation: clear gains on Math and Mix; weaker benefit on TSE (where reasoning signals are more uniformly distributed)
• Related out-of-domain (MMBench): up to +4% improvement
• Generalization benchmarks (MME, MMMU-Pro, MathVista): performance within ±1% of baseline, confirming no catastrophic forgetting
• LLM-as-judge (GPT-4.1 evaluator): consistent gains in logical coherence and explanation clarity for both Qwen2.5-VL-7B and InternVL3-8B

Limitations

• GDP time series is the sole data source for SD-MAR-Math, restricting quantitative reasoning to a single domain.
• InternVL3-8B shows substantially smaller gains than Qwen2.5-VL-7B; the authors attribute this to InternVL3's hybrid architecture limiting steerability under trajectory-level RL.
• BDA provides weaker benefit on TSE tasks where the analytical signal is distributed earlier in the reasoning trace, suggesting BDA's advantage is architecture- and task-dependent.
• Experiments are limited to 7–8B parameter models; scalability to larger or smaller VLMs is not evaluated.
• SD-MAR-TSE images are entirely synthetic (caption-to-image pipeline), which may introduce a domain gap relative to natural-image scenarios like medical imaging or satellite data that motivate the work.

Relevance to Vision-Language Models

SD-MAR directly targets a documented weakness of current VLMs—reasoning across multiple images—and proposes both a scalable data recipe and a targeted RL training modification to address it, making it actionable for practitioners fine-tuning open-source VLMs. The GRPO-lite + BDA formulation is a contribution to the growing literature on RLVR for VLMs (alongside DeepSeek-R1, Dr. GRPO), specifically extending it to multi-input, sequential-reasoning settings. The demonstration that capability-targeted synthetic RLVR can improve a difficult multimodal skill without degrading existing benchmarks is a key empirical result for the VLM training community. The structured perceive→compare→infer framing also provides a useful conceptual scaffold for future multi-image benchmark and dataset design.