Skip to content

Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration

🕒 Published (v1): 2025-05-11 08:32 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

XMedGPT is a clinician-centric multimodal medical AI assistant that extends Generalist Medical AI (GMAI) with visual grounding, sentence-level uncertainty quantification, and prognostic modeling. Trained on 7M image-text pairs (1.6M with pixel-level annotations) across 40 imaging modalities, it delivers interpretable, spatially-grounded outputs with a reliability index to flag uncertain predictions. On 23 public benchmarks and extensive in-house clinical data, it consistently surpasses both specialized GMAI models and GPT-4o.

Problem

Existing GMAI systems lack (1) multimodal explainability—they cannot visually ground their text outputs in specific image regions, preventing clinicians from verifying AI reasoning; (2) prognostic capability—they cannot integrate longitudinal, multi-source patient data for survival or recurrence prediction; and (3) rigorous external validation on heterogeneous real-world clinical data across diverse institutions and diseases.

Method

XMedGPT builds on an 8B-parameter MLLM architecture with three main technical additions:

Visual grounding: A five-step chain-of-thought reasoning pipeline (anatomical identification → region localization → region-specific description → holistic analysis → report synthesis) links text statements to bounding-box coordinates in the image. Fine-tuned via supervised SFT (10,000 MIMIC-CXR samples) followed by DPO on 8,201 high/low-scoring report pairs.

Reliability Indexing: Decomposes each generated sentence into VQA pairs (presence/location/severity of lesions), re-queries the model on the image, and computes semantic entropy over the consistency distribution between extracted attributes and image-grounded answers. Outputs three reliability levels (high/medium/low) per sentence to flag uncertain claims for clinician review. Reformulates standard semantic entropy to account for response confidence, addressing overconfidence from limited sampling in large MLLMs.

Prognostic modeling: Integrates CT/MRI imaging features, TNM staging, histopathological subtypes, patient demographics, and treatment parameters into a unified predictive framework for PFS and OS. Trained on multicenter datasets and externally validated across separate institutions.

Training data: 5,486,115 image-level and 1,686,677 pixel-level image-text pairs spanning 114 open-source datasets, 40 modalities, and 141 anatomical regions.

Key Contributions

  • Unified GMAI framework with coupled visual grounding (bounding boxes linked to generated text) and textual rationale, achieving IoU 0.703 across 141 anatomical regions
  • Entropy-based Reliability Indexing mechanism for sentence-level uncertainty quantification in both VQA and radiology report generation
  • Extension of GMAI to prognostic tasks: PFS prediction for TKI-treated lung cancer, OS for NSCLC, OS for glioblastoma, integrating longitudinal multimodal clinical data
  • Audio modality integration enabling voice-driven clinical interaction
  • Guardrail capability: detects 83.02% of "mismatch scenarios" (query asks for a structure absent from the image) vs. 0% for prior GMAI models
  • Rigorous external validation on 11,530 in-house samples across 11 disease types and 4 anatomical regions from multiple hospitals

Results

Benchmarking (23 public benchmarks): - Unified F1 of 0.639 across five tasks vs. MedRegA (40B) SOTA of 0.529 (+20.7%), with only 8B parameters - Single-label diagnosis: Macro-F1 0.821 vs. MedRegA 0.479 - Multi-label diagnosis: Macro-F1 0.262 vs. prior best 0.126 - VQA: SLAKE F1 0.891, PathVQA F1 0.634, VQA-RAD F1 0.628; SLAKE Closed-Acc 90.95%, PathVQA Closed-Acc 92.39% - OmnimedVQA (42 datasets): 82.44% accuracy vs. MedDr 72.54% - GMAI-MMbench (284 datasets, 38 modalities): outperforms GPT-4o by 29.1%; exceeds >50% on 8 of 18 tasks - CXR captioning: CheXpert-F1 0.479 (MIMIC-CXR, +14.3%) and 0.569 (IU-Xray, +4.6%)

Prognostic performance: - Lung PFS prediction: AUC 0.725 vs. prior SOTA 0.621 (+16.7%) and GPT-4o 0.620 (+25.7% in accuracy); statistically significant Kaplan-Meier stratification on both external cohorts (P<0.001, P=0.008) - NSCLC OS prediction: AUC 0.706 vs. leading clinical risk score 0.598 (+18.1%) - Glioblastoma OS prediction: AUC 0.720 vs. baseline 0.701 (+2.7%); GPT-4o accuracy 0.509 vs. XMedGPT 0.652 - Overall prognostic improvement over prior SOTA: 26.9%; over GPT-4o: 25.0%

Interpretability: - Visual grounding IoU 0.703 vs. MedRegA 0.309; region recognition F1 0.774 vs. MedRegA 0.359 - Kendall's tau-b 0.479 (P<0.05) between visual rationales and clinical outcomes - Uncertainty AUC 0.862 on VQA (SLAKE), 0.764 on report generation (MIMIC-CXR, 5 questions); 9.40% avg improvement over standard semantic entropy

External/in-house validation (11,530 samples): - Multiple-choice accuracy 94.0% vs. MedRegA 80.6%; Diagnosis F1 0.768 vs. MedRegA 0.189; Localization IoU 0.431 vs. MedRegA 0.221 - Outperforms top GMAI by 16.7% overall

Limitations

  • Pulmonary embolism detection shows significant performance drop on external data (IoU drop of 0.44), indicating insufficient generalization for small target detection tasks
  • Prognostic gains on glioblastoma OS are modest (+2.7% AUC over baseline), suggesting limits of single-visit MRI-based survival modeling
  • Reliability Indexing via semantic entropy is computationally expensive due to repeated MLLM sampling; exact inference overhead is not quantified
  • Region recognition slightly lags on external data for kidney and prostatic cancers (robustness issues for certain organ types)
  • Audio input slightly underperforms text input on most benchmarks (mean gap of 0.044 on multi-label diagnosis)
  • Prognostic models trained on relatively small multicenter datasets (n=323–585 training); external validation cohorts are limited in size (n=49–163)

Relevance to Foundation Models in Medicine

XMedGPT directly addresses the clinician trust gap in medical foundation models by coupling generalist perception with spatially-grounded visual rationales and uncertainty-flagged text—two capabilities absent from prior GMAI systems like MedRegA or MedDr. Its extension of GMAI to prognostic tasks (survival, recurrence) represents a meaningful expansion of the functional scope of medical MLLMs beyond image classification and VQA. The consistency-entropy reliability mechanism offers a practical, training-free uncertainty quantification approach transferable to other medical MLLMs. The guardrail and mismatch-detection capability is particularly relevant for safe deployment, a key unsolved problem in deploying foundation models in high-stakes clinical settings.