Skip to content

Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Multimodal LLMs (MLLMs) fail on fine-grained visual differences—subtle changes in object count, position, attributes—because training data lacks controlled minimal-edit pairs and objectives don't enforce feature-level stability across small perturbations. This paper introduces MED (Micro Edit Dataset), a 50K-pair synthetic dataset built via Gemini Flash 2.0 editing guided by Qwen-2.5-VL-72B, plus an SFT loss that penalizes representational drift under small edits. Fine-tuned 7B open-source models match or exceed GPT-4o and Claude 3.7 Sonnet on the companion MED benchmark.

Problem

State-of-the-art MLLMs hallucinate on fine-grained visual changes (object presence, count, spatial relation, attribute) because (1) web-crawled training data lacks tightly controlled image pairs with minimal semantic differences and aligned contrastive captions, and (2) standard autoregressive + CLIP losses do not explicitly enforce that visual embeddings remain stable under small, semantically meaningful edits. Existing benchmarks emphasize coarse-grained tasks and do not systematically isolate these failure modes.

Method

Data construction (MED pipeline): 1. Sample image-text pairs from DOCCI and Visual Genome; filter with Qwen-2.5-VL-72B for quality, caption clarity, and edit potential. 2. Generate edit instructions in 11 hallucination-prone categories (object, attribute, scene, spatial, action, part, counting, comparison, differentiation, negation, universality) using Qwen-2.5-VL-72B. 3. Apply Gemini Flash 2.0 to produce minimally edited counterpart images. 4. Discard pairs with CLIP cosine similarity < 0.7 (clip-vit-base-patch32) to retain only subtle edits. 5. Align captions via a four-step pipeline: complete original caption, generate edited caption, produce difference description (Qwen3-32B), and human-verify 1,000 samples with iterative prompt refinement.

SFT objective: Given an original image \(x_i\) and its edited counterpart \(\hat{x}_i\), the loss is the captioning cross-entropy applied to the feature-space difference \(I_\theta(x_i) - I_\theta(\hat{x}_i)\) against the ground-truth difference description \(S_\phi(t_i, \hat{t}_i)\). This implicitly acts as a feature-level consistency regularizer—forcing the encoder to produce stable embeddings for unmodified regions and sensitive embeddings for changed regions. Joint ViT + LLM fine-tuning is performed via LLaMA-Factory.

Benchmark (MED): 165 multiple-choice questions (15 per edit type) on synthetic pairs with CLIP similarity > 0.95; augmented with 50 real-world pairs from MMVP (MED-Real, 215 total) to test generalization beyond synthetic edits. GPT-4o generates correct answers and three distractors; all items are manually verified.

Key Contributions

  • Semantically controlled image editing pipeline producing minimally different image pairs with aligned contrastive captions at scale.
  • Micro Edit Dataset (MED): 50K+ image-text pairs across 11 fine-grained edit categories, publicly released.
  • MED Benchmark: 215-item evaluation suite (165 synthetic + 50 real-world) for fine-grained visual difference detection.
  • SFT loss framed over feature-space image differences, promoting representational stability and sensitivity to minimal edits.
  • Empirical demonstration that 7B open-source models fine-tuned on MED match or surpass closed-source models (GPT-4o, Claude 3.7 Sonnet) on fine-grained difference tasks.

Results

MED benchmark (accuracy): - Human: 95.21%; GPT-4.1: 54.26%; GPT-4o: 51.32%; Claude 3.7 Sonnet: 40.44% - Qwen2.5-VL-7B (base → ours): 40.24% → 51.61% (+11.37 pp); surpasses GPT-4o and Claude 3.7 Sonnet - Qwen2-VL-7B: 38.48% → 47.55% (+9.07 pp) - LLaVA-V1.6-7B: 40.44% → 44.04% (+3.60 pp) - LLaMA-3.2-11B: 34.71% → 40.92% (+6.21 pp)

MED-Real (real-world pairs): - Qwen2.5-VL-7B: 66.0% → 74.0%; LLaVA-V1.6-7B: 38.0% → 50.0%; LLaMA-3.2-11B: 36.0% → 50.0%

Standard benchmarks (Qwen2.5-VL-7B): - POPE: 96.29 → 97.52; MME perception: 1685.14 → 1701.87; BLINK Count: 50.00 → 59.17; BLINK Visual Correspondence: 33.72 → 37.79 - MMStar Coarse: 73.95 → 75.97; slight drop on Fine-grained Perception (57.35 → 59.36 for this model, marginal decreases on some sub-tasks in other models)

Ablation: ViT-only fine-tuning adds solid gains; joint ViT+LLM fine-tuning is strictly better on average and MME.

Limitations

  • Dataset covers only binary (pairwise) edits; multi-step, temporal, and compositional edit chains are out of scope.
  • Synthetic training data and benchmark originate from the same controlled pipeline, introducing potential distributional overlap despite anti-contamination filtering (MED-Real partially mitigates this).
  • Full ViT + LLM fine-tuning is computationally expensive; parameter-efficient adaptation (LoRA, adapters) is not explored.
  • Framework has not been applied to unified-token architectures (Chameleon, Emu3); transferability is unverified.
  • Minor regressions in fine-grained perception sub-tasks observed for some model/benchmark pairs after fine-tuning.

Relevance to Vision-Language Models

This paper directly targets a well-documented failure mode in VLMs—hallucination on subtle visual differences—and attributes it to two separable causes (data and objective), each addressed with a concrete artifact (MED dataset + SFT loss). The controlled image-editing pipeline for generating hard-negative training pairs is a broadly applicable recipe for any VLM that needs to learn contrastive visual grounding. The result that a 7B open-source model can match GPT-4o on fine-grained difference detection after targeted fine-tuning has practical implications for the training data composition of future VLMs. The MED benchmark also fills a gap between coarse VQA benchmarks and adversarial probing suites, offering the community a tool for systematic hallucination analysis across 11 structured edit categories.