Skip to content

Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA

🕒 Published (v1): 2026-06-18 08:49 UTC · Source: Arxiv · Venue: MICCAI 2025 · link

Why this paper was selected

MICCAI 2025; confidence calibration mismatch in medical MLLMs creates direct misdiagnosis risk

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MLLMs deployed for medical Visual Question Answering (VQA) are systematically overconfident, with a large gap between verbalized confidence and actual accuracy. This paper proposes MS-FBI (Multi-Strategy Fusion-Based Interrogation) combined with an auxiliary expert LLM to recalibrate confidence post-hoc. The method reduces Expected Calibration Error (ECE) by an average of 40% across three medical VQA benchmarks and four MLLM backbones.

Problem

Existing confidence calibration methods for LLMs (token-likelihood, temperature scaling, prompting) have not been systematically evaluated in multimodal medical settings. Medical-domain fine-tuned MLLMs (e.g., LLaVA-Med) exhibit severe overconfidence—ECE as high as 44.28%—likely because supervised fine-tuning on domain data distorts confidence expression. No prior work provided a comprehensive empirical study of accuracy–confidence alignment for MLLMs in Medical VQA.

Method

MS-FBI operates as a two-phase prompt-engineering interrogation pipeline inspired by criminal interrogation techniques:

  1. Initial Inquiry (Punishment phase): The MLLM is prompted with the medical VQA question plus a penalty cue: "You will be punished if the answer is wrong but you answer it with high confidence." The model outputs an answer \(A\) and verbalized confidence \(C \in [0,10]\).

  2. Deep Inquiry (Challenge + Explain phase): A follow-up prompt presents a logical challenge ("I do not think your answer is right. Please…") and an explanation request ("Please explain why you give this answer") in sequence. The model produces a rebuttal \(R_\text{mllm}\).

  3. Auxiliary Expert LLM Assessment: The tuple \((Q, A, C, R_\text{mllm})\) is fed to a separate expert LLM (LLaMA-3-Instruct-8B) which outputs a recalibrated confidence \(C_r\) and reasoning \(R_\text{exp}\).

Six strategy combinations (2 punishment states × 3 challenge/explain activations) are evaluated; the Punish+Challenge (Pu.+Ch.) combination proves most broadly effective. The final confidence used is \(C_r\) from the expert LLM.

Key Contributions

  • First comprehensive empirical study of accuracy–confidence alignment in MLLMs on Medical VQA, revealing that domain-specific SFT worsens calibration.
  • MS-FBI: a modular, prompt-only, black-box-compatible calibration framework requiring no model weight access.
  • Auxiliary expert LLM assessment layer that produces interpretable reasoning about MLLM decision biases alongside the recalibrated score.
  • Demonstration that medical-domain fine-tuning (SFT) significantly degrades confidence calibration relative to general-domain MLLMs.

Results

All ECE values are percentages; lower is better. AUROC higher is better.

  • LLaVA-1.5-Med-7B: Vanilla ECE 44.28% → Ours 26.22% (−40.8%); AUROC 49.48% → 55.07%.
  • LLaVA-NeXT-7B: Vanilla ECE 32.70% → Ours 18.07% (−44.7%); AUROC 56.35% → 57.02%.
  • Molmo-7B: Vanilla ECE 27.68% → Ours 19.75% (−28.7%); AUROC 51.92% → 55.28%.
  • MedVLM-R1: Vanilla ECE 25.62% → Ours 14.48% (−43.5%); AUROC 49.64% → 59.32%.
  • Proposed method outperforms all three baselines (Vanilla, Punish, Top-K) on every model–dataset combination.
  • Ablation: Pu.+Ch. is the best single combination for most models; adding Explain does not consistently help and sometimes hurts.
  • Medical-domain MLLMs have substantially higher baseline ECE than general-domain MLLMs, confirming that SFT on domain data impairs calibration.

Limitations

  • Evaluation restricted to closed-ended (multiple-choice) Medical VQA questions; open-ended generation calibration is not addressed.
  • Expert LLM (LLaMA-3-8B) is a relatively small model; calibration quality may depend on expert LLM capacity.
  • Strategy selection (which combination of Punish/Challenge/Explain) is model- and dataset-dependent with no principled rule for choosing, requiring empirical tuning per deployment.
  • Increased computational cost: each query requires multiple LLM inference passes (initial + deep inquiry + expert assessment).
  • Generalizability to general-domain MLLMs noted as unvalidated; authors explicitly flag this as future work.
  • Benchmark limited to 1,179 closed-ended questions across three datasets; breadth may not reflect real clinical diversity.

Relevance to Vision-Language Models

This work directly targets a critical reliability gap in medical VLMs: the disconnect between a model's expressed confidence and its actual correctness, which is particularly dangerous in clinical decision-support. The finding that domain-specific SFT systematically worsens confidence calibration is a practical warning for the VLM community building specialized multimodal models. The plug-and-play, black-box-compatible MS-FBI framework is broadly applicable to any VLM backbone without retraining, making it immediately relevant to practitioners deploying models like LLaVA-Med or MedVLM-R1. It also raises the open question of how RLHF/GRPO-trained models (MedVLM-R1) compare to SFT models in calibration, with preliminary evidence suggesting RL-trained models start better calibrated.