Skip to content

Benchmarking Foundation Models for Renal Lesion Stratification in CT

🕒 Published (v1): 2026-05-08 13:56 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper benchmarks three open-source medical foundation models (FMCIB, CT-FM, MMM) against a radiomics classifier and a from-scratch 3D ResNet-50 for six-class renal lesion stratification in CT, using a frozen feature-probing protocol. On an external TCIA test set, all foundation model approaches (AUC 0.70–0.77) matched the ResNet baseline (AUC 0.72) but were significantly outperformed by handcrafted radiomics (AUC 0.88). Radiomics remains state-of-the-art for this texture-dependent task.

Problem

Renal lesion subtype classification from CT is clinically impactful but constrained by severe data scarcity—rare subtypes like renal oncocytoma appear in as few as 17 cases in prior work. Standard CNNs trained from scratch underperform in this regime. The question is whether medical foundation model embeddings can substitute large labeled datasets and surpass established baselines, specifically radiomics, in this fine-grained, multi-class setting.

Method

A frozen feature-probing protocol (Strategy B) was applied to three FMs—FMCIB (4096-dim embeddings, 50³ voxel input), CT-FM (512-dim, native sliding-window aggregation), and MMM (1024-dim via max pooling + lesion-area-weighted averaging of 512-dim per-slice features)—plus a 107-feature PyRadiomics pipeline. All encoders fed an XGBoost classifier with class-balanced loss weighting and hyperparameters tuned via 10-fold cross-validation. Additionally, FMCIB was fine-tuned end-to-end (Strategy A) alongside a 3D ResNet-50 from scratch as the deep learning baseline. Training used 2,854 lesions (in-house + KiTS23); external evaluation used 234 lesions from TCIA. Statistical comparison used simultaneous 95% CIs via max-type percentile bootstrap to control family-wise error rate.

Key Contributions

  • First systematic benchmark of three open-source 3D medical FMs (FMCIB, CT-FM, MMM) on CT-based renal lesion subtype stratification with external validation
  • Demonstrates that frozen FM embeddings match from-scratch ResNet performance while eliminating GPU training (feature extraction: ~2.5 CPU-hours vs. 16 GPU-hours; classifier training: 56 seconds/fold)
  • Establishes radiomics as a significantly stronger baseline than all deep learning approaches (∆AUC +0.11 over best FM, 95% CI [0.03, 0.18], p=0.002)
  • Releases RenalVision, an open-source framework unifying the full pipeline with model weights

Results

  • Radiomics (external TCIA, 4-class): AUC 0.88, AP 0.64 — best overall
  • MMM (best FM, external): AUC 0.77, AP 0.50 — statistically inferior to radiomics (p=0.002); not significantly different from ResNet-50 (p=0.307)
  • CT-FM (external): AUC 0.70, AP 0.43 — radiomics significantly superior (p<0.001)
  • FMCIB frozen (external): AUC 0.69, AP 0.40 — radiomics significantly superior (p<0.001)
  • FMCIB fine-tuned (external): AUC 0.70, AP 0.42 — radiomics significantly superior (p<0.001); no benefit over frozen probing
  • ResNet-50 from scratch (external): AUC 0.72, AP 0.45
  • Class-wise (radiomics): Cysts AUC 0.93, ccRCC AUC 0.84, pRCC AUC 0.90, chrRCC AUC 0.86; AP for chrRCC only 0.30 despite high AUC, reflecting class imbalance
  • UMAP shows cysts and ccRCC separate clearly across all embeddings; pRCC, chrRCC, RO overlap substantially
  • Top radiomics features: texture (65% importance), first-order intensity (21%), shape (14%); leading individual features: firstorder_Energy, glcm_Autocorrelation

Limitations

  • External test set excludes RO and "Other" classes (only 4 of 6 classes evaluable externally), complicating full-protocol generalization assessment
  • Dataset over-represents histopathologically ambiguous lesions (selection bias from biopsy/surgery requirement), inflating task difficulty vs. real screening scenarios
  • Fine-tuning evaluated only for FMCIB; no fine-tuning comparison for CT-FM or MMM
  • No consensus on optimal FM downstream protocol (patch size, normalization, aggregation); benchmarks paradigm utility, not definitive FM ranking
  • Low Average Precision across all models (best AP 0.64 for radiomics) precludes autonomous clinical deployment
  • Internal validation metrics may be optimistically biased due to shared use for hyperparameter tuning

Relevance to Foundation Models in Medicine

This is a direct empirical test of whether current generalist 3D medical FMs transfer to a clinically relevant, data-scarce radiology classification task—a central question for the field. The consistent finding that handcrafted radiomics outperforms FM embeddings by a statistically significant margin challenges the assumption that scale and pre-training automatically confer advantages for texture-dependent pathology discrimination. For researchers tracking medical FMs, the result highlights a known gap: FMs pre-trained on large anatomical contexts may not encode the intra-lesion texture heterogeneity needed for histological subtyping, suggesting that task-specific pre-training objectives or fine-tuned feature extractors will be necessary for FM parity with radiomics in such settings. The computational efficiency finding (FM probing matching ResNet with a fraction of the compute) offers a practical interim argument for FM adoption even before accuracy gaps close.