Skip to content

Building a General SimCLR Self-Supervised Foundation Model Across Neurological Diseases to Advance 3D Brain MRI Diagnoses

🕒 Published (v1): 2025-09-12 18:05 UTC · Source: Arxiv · Venue: ICCV 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents a publicly released SimCLR-based self-supervised foundation model for 3D structural brain MRI, pre-trained on 44,958 scans from 18,759 patients across 11 datasets covering multiple neurological diseases. The fine-tuned model outperforms supervised and MAE baselines on all four downstream tasks, including out-of-distribution settings, and retains competitive performance with only 20% of labeled fine-tuning data.

Problem

Existing 3D brain MRI foundation models are individually limited by: (1) narrow training data (one or two pathologies/datasets), (2) low image resolution that blurs clinically relevant small features like lesions, and (3) inaccessibility or non-reproducible preprocessing pipelines. Supervised models are fundamentally constrained by labeled data scarcity, limiting generalizability across neurological conditions and acquisition sites.

Method

SimCLR is extended to 3D by replacing the 2D CNN backbone with a 3D ResNet-18 encoder and adapting augmentations to volumetric data (aggressive random crops, flips, rotations up to 45°, intensity shifts, contrast adjustment via MONAI). Two augmented views per 3D T1 MRI are projected to a 64-dimensional space and trained with NT-Xent contrastive loss (batch size 72, 150 epochs on 12 H100 GPUs). A parallel 3D MAE (ViT backbone, 75% masking, 25×16×16 patches) is implemented as a self-supervised comparison. All inputs are resampled to 1×1×1 mm³ and center-cropped to 150×192×192 voxels using the standardized TurboPrep pipeline (N4 bias correction, skull stripping, linear registration, SynthSeg segmentation, Z-score normalization, ~1 min/scan). Downstream evaluation uses both linear probing and fine-tuning.

Key Contributions

  • High-resolution (1×1×1 mm³) SimCLR foundation model pre-trained on 18,759 patients / 44,958 T1 scans from 11 public datasets spanning Alzheimer's disease, Parkinson's disease, frontotemporal dementia, stroke, MCI, and healthy controls.
  • Systematic comparison of CNN-based SimCLR vs. ViT-based MAE under identical data and evaluation protocols across in-distribution and out-of-distribution tasks.
  • Data-efficiency analysis showing SimCLR-FT with 20% labeled data still outperforms supervised baselines trained on 100%.
  • Full public release of model weights and reproducible preprocessing pipeline (TurboPrep).

Results

  • Stroke scale regression (SOOP, in-distribution): SimCLR-FT MAE 5.37 vs. supervised ResNet-18 5.47, MAE-FT 6.15.
  • Alzheimer's classification (AIBL, out-of-distribution, n=106 train): SimCLR-FT AUC 0.929 vs. supervised ResNet-18 0.869 (+6% AUC); MAE-FT 0.798.
  • Sex classification (IXI, out-of-distribution): SimCLR-FT AUC 0.991 vs. supervised ResNet-18 0.988, ViT-T 0.876.
  • Age regression (IXI, out-of-distribution): SimCLR-FT MAE 4.35 years vs. supervised ResNet-18 4.85, MAE-FT 8.53.
  • SimCLR-FT outperforms all models on all four tasks despite being fine-tuned for only 100 epochs vs. 300 epochs for supervised baselines.
  • With 20% of Alzheimer's training data (10 samples/class), SimCLR-FT still exceeds supervised models using 100% of data.
  • All three ViT-MAE sizes (Tiny/Base/Large) perform similarly, with Tiny selected for efficiency; CNN models consistently outperform ViT models at this dataset scale.

Limitations

  • Dataset scale (~44K scans) remains small relative to natural imaging, which constrains ViT-based approaches and may cap representation quality for rare/subtle pathologies.
  • Only T1 sequences used; other clinically common sequences (T2, FLAIR, DWI) are excluded, limiting applicability to multi-contrast tasks.
  • Out-of-distribution Alzheimer's test sets are severely class-imbalanced (validation/test: 215 healthy vs. 15 AD), making AUC estimates high-variance.
  • AIBL fine-tuning uses only 106 balanced samples; results may not generalize to larger or more heterogeneous AD cohorts.
  • MAE baseline hyperparameters (patch size, masking ratio) were not extensively tuned; ViT underperformance may partly reflect suboptimal configuration rather than architectural inferiority.
  • No evaluation on segmentation or detection tasks, leaving the model's utility for those clinically critical objectives uncharacterized.

Relevance to Foundation Models in Medicine

This work directly addresses the practical gaps—limited data diversity, low resolution, and inaccessibility—that have hampered adoption of 3D neuroimaging foundation models in clinical research. By demonstrating that a contrastive CNN-based SSL model pre-trained on heterogeneous public datasets substantially outperforms both supervised and MAE baselines in out-of-distribution settings with minimal labels, it provides concrete evidence for the value of disease-diverse pre-training over pathology-specific supervised approaches. The public release with a reproducible preprocessing pipeline lowers the barrier to community adoption, a chronic bottleneck in medical foundation model research. For researchers tracking this topic, the CNN-vs-ViT finding under realistic medical data scales is a practically important calibration point, contrasting with the ViT-dominated trends in 2D medical imaging.