Skip to content

Google-MedGemma Based Abnormality Detection in Musculoskeletal radiographs

🕒 Published (v1): 2025-11-06 00:51 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper adapts Google's MedGemma-4B foundation model for binary abnormality detection on the MURA musculoskeletal radiograph dataset by freezing most of its SigLIP vision encoder and appending a lightweight MLP classifier head. The approach achieves overall accuracy 0.92 and AUROC 0.95, outperforming CNN and autoencoder baselines. Selective unfreezing of the top transformer blocks proves more effective than head-only fine-tuning.

Problem

Conventional CNN and autoencoder pipelines for musculoskeletal radiograph abnormality detection require extensive network engineering and large labeled datasets, and struggle to generalize across anatomical sites and imaging protocols. Prior best methods on MURA top out at accuracy ~0.87 and AUROC ~0.93, leaving headroom for more scalable, transferable approaches.

Method

The pipeline extracts fixed-length image embeddings from the MedGemma-4B-PT vision backbone—a 27-layer SigLIP ViT with 1152-dimensional token embeddings, patch size 14×14, processing 896×896 inputs yielding 4096 tokens—then applies mean pooling to produce a single 1152-d vector. A three-layer MLP head (1152→512→128→1, with ReLU, dropout 0.30/0.20, and sigmoid output) performs binary classification. Training uses a selective unfreezing regime: patch/positional embeddings and lower encoder layers stay frozen; the top K∈{2,3,4} transformer blocks and the MLP head are fine-tuned with AdamW under a two-tier learning rate (lower for encoder blocks, higher for head) with cosine decay and warmup. Multi-view studies aggregate per-image probabilities before thresholding.

Key Contributions

  • Application of MedGemma-4B-PT (SigLIP-derived medical vision encoder) to musculoskeletal radiograph classification, replacing bespoke feature engineering with a pretrained medical backbone.
  • Selective encoder unfreezing strategy that preserves general medical priors while enabling task-specific adaptation.
  • Systematic ablation over unfreezing depth (K=2,3,4 blocks) and input resolution (896×896), quantifying the accuracy–efficiency tradeoff.
  • State-of-the-art results on MURA under patient-disjoint splits, with per-anatomy reporting across all seven upper-extremity categories.

Results

  • Overall (proposed): Accuracy 0.92, Precision 0.91, Recall 0.91, F1 0.91, AUROC 0.95
  • Per-anatomy peaks: Humerus F1 0.95/AUROC 0.97; Shoulder F1 0.93/AUROC 0.96; weakest on Wrist F1 0.88/AUROC 0.93
  • vs. DenseNet169 baseline: Accuracy 0.83, F1 0.83 (no AUROC reported)
  • vs. MURA paper original: Accuracy 0.73
  • vs. calibrated deep learning ensemble: Accuracy 0.87, Precision 0.93, Recall 0.81, F1 N/A, AUROC 0.93
  • Selective unfreezing (top K blocks) consistently outperforms head-only adaptation on macro-F1 and AUROC per ablation in Figure 3.

Limitations

  • No external validation dataset; evaluation is confined to MURA, limiting assessment of cross-institutional generalizability.
  • Reported numerical comparison table (Table 2) is sparse—F1 and AUROC are missing for several baselines, weakening head-to-head claims.
  • Only binary (normal/abnormal) classification at study level; no fracture localization or grading.
  • Single GPU experiments; computational cost of 896×896 inputs and 4B-parameter backbone not quantified relative to deployment constraints.
  • No uncertainty quantification or calibration curves reported despite clinical triage framing.
  • Class imbalance handling is mentioned only briefly; no analysis of its effect on per-anatomy thresholds.

Relevance to Foundation Models in Medicine

This paper is a direct probe of MedGemma's practical utility as a feature extractor for a well-benchmarked radiology classification task, providing concrete evidence that a medical VLM's vision encoder—without its language components—transfers effectively to a supervised binary diagnosis setting with minimal fine-tuning. It contributes to the emerging evidence base on parameter-efficient adaptation strategies (selective layer unfreezing) for medical foundation models, complementing work on BiomedCLIP and LLaVA-Med. For researchers tracking foundation models in medicine, the MURA results offer a reproducible reference point for comparing future medical VLMs on musculoskeletal imaging, though the absence of external validation and the narrow task scope limit broader architectural conclusions.