Skip to content

VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context

🕒 Published (v1): 2026-06-29 13:30 UTC · Source: Arxiv · Venue: ECCV 2026 · link

Why this paper was selected

Addresses the long-visual-context bottleneck that limits real-world VLM deployment

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VisReflect improves fine-grained perception in high-resolution images and long videos by generating continuous visual reflection tokens in latent space that recall salient region/frame features, rather than predicting discrete coordinates for cropping. It achieves +4.1% on image benchmarks and +1.8% on video benchmarks over strong baselines, matching zooming-based methods while reducing inference time by ~44% via single-pass forward.

Problem

As visual token counts grow in high-resolution images and long-form videos, the visual attention sink phenomenon worsens — irrelevant tokens absorb disproportionate attention mass, degrading fine-grained perception. Existing "thinking-with-images/videos" approaches mitigate this by predicting bounding boxes or temporal spans and re-encoding cropped regions, but suffer from (i) unreliable numeric localization in discrete token space that doesn't reflect the underlying visual feature space, and (ii) high computational cost from multi-pass forward (re-encoding and re-prefilling).

Method

VisReflect adds three special tokens to the vocabulary: <BOR> (beginning of reflection), <VR> (visual reflection token), and <EOR> (end of reflection). During training, given a query and ground-truth bounding box or temporal span, the model generates a sequence of <VR> tokens whose last-layer hidden states are aligned via cosine similarity to the visual tokens of the region-of-interest (image) or sampled salient frames (video). In visual reflection mode, the hidden state of a <VR> token replaces its own embedding as input for the next decoding step (rather than projecting through the LM head), enabling the model to internally "reflect" relevant visual content in continuous embedding space. The overall loss combines standard cross-entropy for answer generation with an alignment loss \(L_{\text{VR}}\) weighted by \(\lambda = 0.1\). During inference, <BOR> is appended to the question prompt; the model autoregressively generates <VR> tokens (up to a limit), then <EOR>, followed by the answer — all in a single forward pass with no explicit cropping or re-encoding. The Qwen-2.5-VL-7B backbone is trained in two stages (image then video) with frozen vision encoder and projector, updating only the LLM parameters.

Key Contributions

  • Proposes latent visual reflection tokens that operate in continuous embedding space, avoiding unreliable numeric coordinate predictions and multi-pass inference required by prior cropping-based approaches.
  • Introduces a training objective that aligns reflection token hidden states with region-of-interest visual tokens via cosine similarity, enabling the model to attend to question-relevant visual content within a single forward pass.
  • Demonstrates consistent gains on both high-resolution image benchmarks (+4.1%) and long-video benchmarks (+1.8%), with inference time ~44% lower than comparable zooming-based methods.

Results

  • Image benchmarks: +0.6 on BLINK, +2.9 on HRBench-4K, +3.2 on HRBench-8K, +5.8 on V over Qwen2.5-VL-7B backbone. Outperforms single-pass Thyme-VL† by +1.2/+2.3/+4.3 on HRBench-4K/HRBench-8K/V.
  • Video benchmarks: +0.8 on MLVU, +2.0 on MVBench, +2.1/+2.0 on VideoMME (w/o and w/ subtitles) over Qwen2.5-VL-7B.
  • Latency: Matches multi-pass zooming methods (LOVE-R1, VideoChat-R1.5, TimeSearch-R) in accuracy while reducing inference time by ~44% on video (e.g., 18.5s vs 32.5–37.1s on MLVU).
  • Ablation: Cosine similarity alignment outperforms MSE; 128 <VR> tokens is the optimal training length; finetuning the backbone alone yields negligible gains (visual reflection is the main driver).

Limitations

  • Requires bounding box / temporal span annotations during training as supervision for the alignment loss.
  • Performance saturates beyond 128 <VR> tokens; excessively long reflection sequences can degrade results.
  • The number of <VR> tokens during inference is fixed at the training limit (not dynamically determined per query), which may produce redundant or insufficient reflection.
  • Evaluation limited to Qwen-2.5-VL-7B backbone; scalability to larger models or different architectures is unexplored.

Relevance to Vision-Language Models

VisReflect addresses a core bottleneck in LVLMs — attention dilution under massive visual token counts — without the computational overhead of multi-pass cropping pipelines. It unifies high-resolution image and long-video perception under a single mechanism, suggesting that latent-space retrieval of visual evidence can replace explicit tool-use for fine-grained perception. This is particularly relevant as VLMs scale to higher resolutions and longer temporal contexts where token efficiency and attention fidelity become critical.