Skip to content

MiCo: Multi-image Contrast for Reinforcement Visual Reasoning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MiCo trains VLMs to reason across multiple images using self-supervised contrastive triplets — no human-annotated QA pairs required. It constructs triplets of two augmented views of the same image plus a visually similar but distinct image, then applies rule-based RL (Augmented GRPO) to teach the model to compare fine-grained visual differences. The resulting multi-image reasoning ability generalizes broadly to held-out benchmarks without task-specific supervision.

Problem

Current VLMs fail at tasks requiring cross-image visual linking — e.g., tracking object identity across views, detecting subtle state changes, inferring camera motion — because they lack the meta-skill of visual comparison. Existing RL-based reasoning approaches (GRPO, MM-Eureka, NoisyRollout) require hand-crafted or model-generated QA pairs, which are expensive to produce for fine-grained multi-image scenarios.

Method

Data construction: Image pairs \((I_a, I_b)\) are drawn from two sources: (1) video frames sampled 2 seconds apart, filtered by SSIM to avoid near-identical pairs; (2) image editing datasets ("before/after" pairs), filtered by MSE. Each pair is augmented via random crop+resize to produce training triplets \(T = \{T_1(I_a), T_2(I_a), T_3(I_b)\}\), where the first two images are the same and the third is different.

Task formulation: The VLM is prompted to output a CoT <think> block and answer three binary same/different comparisons (image1 vs image2, image2 vs image3, image1 vs image3), encoded as a 3-bit string (e.g., TFT). QA pairs also include image-pair comparisons and 20–50 GPT-4o-generated prompt paraphrases for diversity.

Augmented GRPO: Instead of sampling rollouts and optimizing on the same inputs (as in standard GRPO), MiCo samples \(G=8\) trajectories under weak augmentations \(T^w\) (easier), then optimizes the policy on strong augmentations \(T^s\) (harder). Rewards use a binary format reward (tag compliance) and accuracy reward (all three comparisons correct), weighted 1:1. KL regularization (\(\beta=0.01\)) anchors to the reference policy.

Key Contributions

  • Self-supervised contrastive triplet construction from video frames and image editing data — eliminates need for human/model-annotated QA pairs.
  • Augmented GRPO: decouple rollout sampling (weak augmentation) from policy optimization (strong augmentation), transferring high-quality CoTs to harder visual inputs.
  • Demonstration that a model trained only on same/different image comparisons generalizes to diverse multi-image and single-image benchmarks.
  • Ablation of training paradigm (SFT vs. no-CoT RL vs. CoT RL), data source, rollout augmentation strategy, sample formulation, prompt diversity, and augmentation type.

Results

  • VLM2-Bench (primary): MiCo-7B-CoT achieves 61.06% overall average, +12.93 points over baseline Qwen2.5-VL-7B (48.13%), outperforming GPT-4o (60.36%) and all 7B reasoning baselines (MM-Eureka: 57.24%, NoisyRollout: 50.08%, ThinkLite: 55.79%, VLAA-Thinker: 58.49%).
  • Largest gains on General Mat (+13.90), General Trk (+24.20), General Grp (+22.00), Object Grp (+23.00) sub-tasks.
  • MuirBench: 60.53% vs. 58.43% baseline (+2.10); competitive with MM-Eureka (60.57%).
  • BLINK: 57.23% vs. 55.54% baseline (+1.69).
  • HallusionBench: 69.61% (+0.11); MMStar: 65.60% (+1.54); MMMU: 54.77% (+0.66); MathVista: 67.90% (+0.80) — modest single-image gains.
  • MiCo excels on Visual Retrieval, Semantic Correspondence, Spatial Relation sub-tasks; underperforms on Scene Understanding, Forensic Detection, and Relative Depth.
  • CoT reasoning hurts person-track tasks (face identity), hypothesized because facial nuances are hard to verbalize.

Limitations

  • Weak on tasks requiring domain-specific priors: face verification, visual math, forensic detection, relative depth estimation.
  • CoT reasoning provides limited or negative gains for person-centric identity tasks (facial nuance is not easily verbalized).
  • 12.5% random-guess rate for triplet T/F task introduces noise; confidence-based reweighting failed to address this.
  • Importance sampling to correct distribution shift between weak/strong augmentation rollouts did not improve results and may destabilize GRPO's internal ranking.
  • No geometric or depth-specific training signal — spatially grounded reasoning remains limited.
  • Training data restricted to OmniEdit and VidGen-1M; generalization to other editing styles is demonstrated but not exhaustively validated.

Relevance to Vision-Language Models

MiCo directly targets a known VLM failure mode — cross-image visual reasoning — and proposes a scalable, annotation-free training paradigm that eliminates the bottleneck of curated multi-image QA data. The Augmented GRPO strategy is a transferable technique for other RL-based VLM training scenarios where hard instances are difficult to sample rollouts from directly. The result that training on a single abstract task (image comparison) transfers to diverse downstream benchmarks strengthens the case for meta-skill learning as a training objective for VLMs, complementing task-specific fine-tuning approaches. This work fits within the rapidly expanding line of RL-for-VLM reasoning (GRPO, NoisyRollout, MM-Eureka) and is the first to derive reward signals from inherent visual structure rather than human or model annotations.