Skip to content

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

🕒 Published (v1): 2026-06-25 17:59 UTC · Source: Arxiv · Venue: ECCV 2026 · link

Why this paper was selected

ECCV 2026; self-evolving LMM training with visual token attention — unsupervised reasoning improvement

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Self-evolving LMMs that optimize for answer consistency suffer from visual under-conditioning: the decoder relies on language priors rather than image evidence, hurting captioning and VQA. VISE fixes this with two unsupervised invariance-based rewards—geometric and semantic—that directly regularize how much the decoder attends to visual tokens. Trained on 4000 raw unlabeled COCO images, VISE achieves +16.85 CIDEr on COCO and +19.66 CIDEr on TextCaps with no regressions across any benchmark.

Problem

Existing self-evolving LMMs (EvoLMM, iReasoner, VisPlay) use multi-role self-play and self-consistency rewards that optimize answer agreement. Because a language-prior-driven decoder can produce self-consistent outputs without attending to the image, these methods reinforce visual under-conditioning rather than correcting it. This causes hallucination, modality bypass, and degraded performance on captioning and region-level understanding tasks even while improving structured reasoning scores.

Method

VISE is a single-model, fully unsupervised REINFORCE framework trained on raw unlabeled images. The model self-generates a natural-language localization query \(q\) and bounding box \(B_{\text{orig}}\) for each image; both are produced by the same policy \(\pi_\theta\) with no separate roles.

Geometric Invariance Reward. A spatial transformation \(T\) (affine, crop, or flip) described by homogeneous matrix \(M\) is applied to the image. The model predicts \(B_{\text{new}}\) on the transformed view; the expected box is computed by projecting \(B_{\text{orig}}\) corners through \(M\). The reward is: $\(R_{\text{geo}} = \frac{\text{GIoU}(B_{\text{proj}}, B_{\text{new}}) + 1}{2}\)$ where GIoU penalizes spatial inconsistency across views, directly targeting the localization instability that characterizes visual under-conditioning.

Semantic Invariance Reward (Ghosting). The pixel region inside \(B_{\text{orig}}\) is replaced by a Gaussian-blurred version (\(\sigma=25.0\)), producing a ghosted image \(\tilde{x}\). The model judges object visibility on both \(x\) and \(\tilde{x}\). The reward is: $\(R_{\text{sem}} = \begin{cases} 1.0 & \text{if } v(x,q)=1 \text{ and } v(\tilde{x},q)=0 \\ 0.0 & \text{otherwise} \end{cases}\)$ This penalizes evidence-agnostic generation: a model driven by language priors will still claim the object visible after the region is blurred.

Optimization. Total reward \(R_t = 0.5\,R_{\text{geo}} + 0.5\,R_{\text{sem}}\) with an EMA baseline for variance reduction. The loss is KL-regularized REINFORCE against a frozen reference policy with adaptive KL coefficient \(\beta_t\) (target divergence \(\tau=0.02\), adaptation rate \(\eta=0.1\)). The vision encoder is frozen throughout; only the multimodal projector, feed-forward layers, and decoder attention projections are updated via LoRA.

Key Contributions

  • Visual under-conditioning diagnosed as the structural failure of answer-agreement rewards in self-evolving LMMs, with direct evidence from attention analysis and CIDEr regressions in prior methods.
  • Geometric invariance reward: self-supervised signal using known spatial transformations to enforce localization consistency across views.
  • Semantic invariance reward via ghosting: self-supervised signal that requires the model to detect evidence removal after regional blurring, preventing evidence-agnostic generation.
  • Single-model framework: eliminates Proposer–Solver minimax instability; no annotations, metadata, or external reward models.
  • Evaluated across 18 benchmarks and four Qwen3-VL scales (2B, 4B, 8B, 32B) with no tradeoffs between task groups.

Results

  • COCO captioning (2B): +16.85 CIDEr (21.54 → 38.39); best baseline (VisionZero-RW) achieves only +4.04.
  • TextCaps (2B): +19.66 CIDEr (22.20 → 41.86); best baseline +3.08.
  • NoCaps (2B): +14.73 CIDEr; Flickr30k (2B): +16.55 CIDEr.
  • Hallucination (COCO Cap Chair-I, 2B): −5.0 points reduction.
  • COCO captioning (32B): +8.72 CIDEr vs. base; best baseline (VisionZero-RW) +7.76.
  • Visual attention: mean generation-time visual token attention increases +2.84% (2B) and +2.56% (4B) across mid-to-late decoder layers 15–25, with per-sample peaks up to +5.09%.
  • CKA analysis: VISE increases cross-view representational alignment in final decoder layers (2B peak \(\Delta=+0.069\) at layer 27; 4B peak \(\Delta=+0.253\) at layer 33).
  • Prior methods EvoLMM and iReasoner regress on captioning (−0.70 and −0.61 CIDEr on COCO at 2B), confirming visual under-conditioning is reinforced by answer-consistency training.
  • VQA and reasoning benchmarks show consistent improvement with no tradeoffs across all scales.

Limitations

  • Training uses only 4000 COCO images; generalization to out-of-domain distributions relies on supplementary Objects365 experiments but is not characterized at scale.
  • Ghosting reward is binary (\(R_{\text{sem}} \in \{0,1\}\)), providing no gradient signal for partially informative perturbations.
  • Geometric invariance reward requires spatially unambiguous objects; queries for textures or full-scene descriptions may not produce meaningful bounding boxes.
  • Semantic invariance reward requires the model to already partially localize correctly; at the start of training, collapsed localization could prevent the ghosting signal from being informative.
  • Vision encoder is frozen, so the method addresses only decoder-side visual under-conditioning; encoder representation failures are out of scope.
  • Evaluation is restricted to the Qwen3-VL family; cross-architecture generalization to other backbones is not demonstrated in the main paper.

Relevance to Vision-Language Models

VISE directly addresses a failure mode that is increasingly important as self-supervised and self-evolving training replaces human annotation in LMM post-training: models can score well on reasoning benchmarks while their decoder silently bypasses visual content. The invariance-based reward design offers a clean, annotation-free diagnostic and fix specifically for the multimodal conditioning gap, as opposed to improving language reasoning quality. This connects to the broader VLM literature on hallucination mitigation, attention sink phenomena in transformer decoders, and the distinction between visually-grounded versus language-prior-driven generation—an open problem for all instruction-tuned VLMs trained with RLHF-style objectives.