Skip to content

Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

🕒 Published (v1): 2025-11-14 11:27 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MUG (Multi-agent Undercover Gaming) reformulates Multi-Agent Debate (MAD) as a social deduction game where one agent is assigned a counterfactually edited image and must be identified and eliminated before the remaining agents collaboratively answer the question. By using ground-truth counterfactual image edits to expose hallucinating agents, MUG replaces unreliable statistical consensus with factual verification, achieving consistent gains over MAD baselines on multimodal reasoning and hallucination benchmarks.

Problem

Standard MAD protocols (majority voting, super-judge) assume all debating agents are rational and self-correcting — an assumption that breaks down when agents themselves hallucinate. Hallucinating agents can form a majority or go undetected, corrupting consensus. No prior MAD work provides a ground-truth mechanism to identify which specific agents are generating incorrect outputs rather than trusting statistical agreement.

Method

MUG operates as a structured two-phase game within a system S = (Q, A, F, R):

  1. Counterfactual image generation: Given a question Q and factual image I⁺, an LLM classifies the question type (e.g., "How Many" → Quantity Editing) and identifies target objects via scene graph parsing. Step1X-Edit then generates I⁻ subject to three constraints enforced by a weighted score: maximal visual similarity (ViT embeddings), semantic consistency (CLIP similarity), and naturalness (FID). The edited image is accepted only if α·C_vs + β·C_sc + γ·C_na ≥ threshold.

  2. Undercover Detection Game: One agent is assigned I⁻ (the "undercover"); all others receive I⁺. Agents reason about Q, strategically revealing or concealing which image they hold. Each agent votes on who is the undercover using four factors: inconsistency score, deviation from consensus, detail accuracy, and behavioral suspicion (vague/defensive language). Elimination follows majority vote; the game runs until the undercover is voted out or termination conditions are met (too few agents, timeout).

  3. Summarization Game: After the undercover agent is eliminated, surviving agents (all holding I⁺) collaboratively produce a final answer via structured dialogue, pooling collective reasoning from prior rounds.

Key Contributions

  • Factual verification over statistical consensus: Counterfactual edits provide ground-truth labels for which agent is hallucinating, replacing reliance on majority agreement.
  • Dynamic cross-evidence reasoning: Multiple evidence sources (I⁺ and I⁻) created per-query replace the static single-image input of prior MAD.
  • Active probing: Agents must pose questions and identify discrepancies rather than passively answering, forcing detail-oriented reasoning.
  • Automatic termination criterion: Game ends naturally when the undercover is voted out, unlike MAD which relies on a fixed round maximum or super-judge.
  • Image generation quality control: Three-constraint generation pipeline (visual similarity, semantic consistency, naturalness) ensures counterfactual images are realistic yet semantically distinct.

Results

  • MMMU: Qwen2.5VL-7B+MUG achieves 50.3% (+5.3pp over base, +5.6pp over MAD-Vote, +2.9pp over MAD-Judge); InternVL3-14B+MUG achieves 60.7% (+0.9pp over base, +5.5pp over MAD-Vote).
  • HallusionBench average: Qwen2.5VL-7B+MUG achieves 53.8% vs 37.8% for MAD-Vote (+16.0pp) and vs 50.2% for MAD-Judge (+3.6pp); InternVL3-14B+MUG achieves 58.0%, best across all baselines.
  • POPE: Qwen2.5VL-7B+MUG achieves 88.4% F1 (vs 74.1% MAD-Vote, 83.5% MAD-Judge); InternVL3-14B+MUG achieves 91.1% F1 (vs 88.8% MAD-Vote, 90.6% MAD-Judge).
  • MMStar: Qwen2.5VL-7B+MUG achieves 63.8%, InternVL3-14B+MUG achieves 69.1%, both best among their respective ablation groups.
  • Qwen2.5VL-7B+MUG surpasses GPT-4v and Claude 3.5 Sonnet on MMStar (63.8% vs 62.2% and 63.8% vs 65.9% respectively — competitive or better).
  • Ablation: removing counterfactual editing drops HallusionBench by 3.61pp; removing the undercover mechanism drops it by 4.49pp.
  • Performance peaks at round 1; additional rounds yield diminishing returns.
  • All improvements are statistically significant (one-tailed paired t-test, p < 0.01).

Limitations

  • Requires an external image editing model (Step1X-Edit) with a non-trivial quality-controlled generation loop, adding inference overhead and pipeline complexity.
  • Performance degrades with additional game rounds beyond round 1; optimal round count is task- and dataset-dependent.
  • The undercover detection mechanism may fail in later rounds for difficult samples, with ~5% undercover survival rate in rounds 3–4.
  • The counterfactual image generation assumes the LLM can reliably classify question type and extract edit targets; failure here would produce uninformative counterfactuals.
  • Evaluated only on Qwen2.5VL-7B and InternVL3-14B as the primary MUG-augmented models; broader backbone coverage is absent.
  • The framework does not straightforwardly extend to text-only or video-only domains without modality-specific counterfactual generation pipelines.

Relevance to Agentic AI / LLM Agents

MUG directly addresses a core failure mode in multi-agent LLM systems: hallucinating agents corrupting group outputs through unchecked consensus. The social deduction game framing is a novel structural primitive for agent orchestration — using information asymmetry and adversarial probing to enforce factual grounding rather than trusting agent self-assessment or majority vote. This connects to a broader line of work on agent reliability, debate-based verification (MAD), and self-refinement, offering a concrete mechanism for identifying which agent in a pipeline is producing incorrect outputs. For researchers building multi-agent reasoning systems, MUG demonstrates that structured game-theoretic protocols with ground-truth verification signals can substantially outperform classical debate approaches, particularly for hallucination-sensitive tasks.