Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Current VLMs trained with reinforcement learning (RL) or supervised fine-tuning (SFT) exhibit a significant compositional ability gap: they perform well on individual tasks but fail to integrate skills across modalities or reasoning domains. RL consistently outperforms SFT on compositional generalization, but both strategies remain far from sufficient for multimodal compositional reasoning. The proposed RL-Ground method—combining caption-before-thinking with intermediate progress rewards—substantially narrows this gap.

Problem¶

It is unknown whether RL-based post-training strategies that improve LLM reasoning (e.g., DeepSeek-R1, GPT-o1) transfer compositional generalization to VLMs. Specifically: (1) can pure-text reasoning skills compose with visual perception at inference? (2) can independently learned visual sub-skills integrate into composite tasks? (3) does this generalize OOD? Standard VLM post-training evaluations use isolated benchmarks and do not probe cross-modal or cross-task skill composition.

Method¶

The authors introduce ComPABench, a controlled diagnostic benchmark with three task families: - Cross-modal composition (RQ1): Models train on pure-text geometric/spatial reasoning, then are evaluated on image-format variants of the same tasks. - Cross-task composition (RQ2): Models train on geometric reasoning (shape area) and spatial reasoning (grid position) independently, then are evaluated on compositional tasks requiring both simultaneously (e.g., "find the nearest shape and compute the combined area"). - OOD generalization (RQ3): Objective modifications (e.g., largest area instead of total area, farthest instead of nearest shape) in both individual and compositional settings.

Training paradigms compared: SFT, RL (GRPO with answer-correctness + format reward), and SFT-initialized RL, on Qwen2.5-VL-3B and 7B backbones. 4K training samples per task type, 500 evaluation samples.

RL-Ground is a proposed remedy: it enforces a <caption> block before <think>, compelling the model to verbalize visual content first, and adds a progress reward that scores intermediate grounding steps (e.g., correct area of individual shapes) in addition to final-answer accuracy.

Key Contributions¶

ComPABench: a diagnostic benchmark for probing cross-modal, cross-task, and OOD compositional generalization in VLMs.
Systematic empirical comparison of SFT, RL, and SFT-init RL across all three compositional axes, revealing that SFT causes catastrophic forgetting of compositional ability while RL preserves it.
Identification of two key ingredients for VLM compositionality: visual-to-text alignment and accurate visual grounding.
RL-Ground: a training strategy combining caption-before-thinking and step-wise progress rewards, achieving best performance across in-distribution and OOD compositional tasks.

Results¶

Cross-modal gap (RQ1): SFT trained on pure-text reaches ~99% on text tasks but collapses to 13–16% on multimodal variants (>94-point drop). RL closes part of this gap (multimodal shape area: 28% vs. 16.2% for SFT 7B) but remains far below text-modality performance. Initializing multimodal RL from a pure-text checkpoint boosts grid-position multimodal accuracy from 49.6% → 64.4% (3B).
Cross-task composition (RQ2): SFT on individual tasks destroys baseline compositional ability (49.4% → 0.6% for 3B pure-text). RL recovers this strongly (93% 3B, 81.2% 7B pure-text), but multimodal RL only reaches 17.4% (3B) / 31.2% (7B). RL-Ground reaches 52.8% (7B), the best multimodal compositional result.
OOD generalization (RQ3): SFT fails OOD compositional tasks (1.2–2.8%). RL generalizes to OOD individual tasks (95.8% shape area OOD on 3B) but is weak on OOD compositional tasks at 3B (6.6%), better at 7B (40.4%). RL-Ground achieves 38% (3B) / 52.8% (7B) on OOD compositional tasks.
Grounding tasks: RL-Ground achieves 96.2% Shape Area Grounding and 88.6% Grid Position Grounding (7B), outperforming SFT (1.2% / 65.4%) and standard RL (96.6% / 74.8%).
Real-world transfer (Geometry3K zero-shot): RL-Ground 21.2% vs. base 14.8%, SFT 9.2%, RL-only 17.8%.

Limitations¶

ComPABench is synthetic (grid-based geometric/spatial tasks); real-world compositional diversity may expose different failure modes not captured here.
Experiments limited to Qwen2.5-VL-3B and 7B; behavior may differ at larger scales or with different VLM architectures.
RL-Ground's progress reward requires task-specific intermediate supervision signals, which may be difficult to define for open-ended or less structured VQA tasks.
SFT-init RL for compositional tasks remains near-zero (1.0–2.6%), and the paper's proposed fix (RL-Ground) is evaluated only without SFT initialization—the interaction is not fully ablated.
The caption-before-thinking prompt format is manually enforced; it is unclear how this interacts with the model's natural reasoning under distribution shift.

Relevance to Vision-Language Models¶

This paper directly targets a core open question in VLM post-training: whether RL-based strategies (analogous to those driving LLM reasoning gains) confer generalizable compositional reasoning across modalities. The finding that SFT catastrophically destroys baseline compositional ability while RL preserves it has immediate implications for practitioners choosing post-training recipes. The RL-Ground strategy—grounding via captioning before reasoning plus dense intermediate rewards—offers a concrete, reproducible technique for improving cross-modal and cross-task generalization. This connects to active threads in VLM research around visual grounding fidelity, process reward models for multimodal tasks, and the limits of outcome-only reward signals for spatially complex reasoning.