Skip to content

ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

🕒 Published (v1): 2026-07-10 15:35 UTC · Source: Arxiv · link

Why this paper was selected

Pathology foundation model fusing vision, VL, and slide-level experts; builds on strong computational pathology trend

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ALICE is a unified pathology foundation model trained via multi-stage agglomerative distillation, sequentially absorbing knowledge from eight specialized teacher models (three vision-only, three vision-language, two slide-level) into a single backbone. Pretrained on ~25M tile images and 155K high-resolution WSIs, it achieves the best average rank across 96 downstream tasks spanning ROI analysis, vision-language multimodal tasks, and whole-slide clinical assessment. The approach demonstrates that hierarchical distillation across modality and scale boundaries consolidates fragmented expertise into a single general-purpose representation.

Problem

Existing pathology foundation models (PFMs) are specialized by pretraining paradigm: vision-only models (UNI-2, Virchow-2) capture dense morphology but lack language alignment; vision-language models (CONCH, MUSK, KEEP) align with text but underperform on fine-grained visual discrimination; slide-level models (GigaPath, TITAN, CARE) encode whole-slide context but have limited transferability. No single model covers the full spectrum of morphological, semantic, and slide-level capabilities required across clinical pathology tasks, and prior unification attempts (GPFM) do not explicitly organize heterogeneous knowledge by modality, scale, or clinical abstraction level.

Method

ALICE employs a multi-stage agglomerative distillation framework over a single ViT-like backbone partitioned into three transformer modules: vision-only, multimodal, and slide-level.

Stage 1 — Vision-only: The patch-embedding module and vision-only transformer are trained to align with ensemble features from UNI-2, Virchow-2, and H-Opt-1 using ~25M low-resolution tile images. The multimodal and slide-level transformers remain inactive.

Stage 2 — Multimodal: The visual backbone is frozen. The multimodal transformer is trained to align with language-paired representations from CONCH, MUSK, and KEEP, inheriting already-distilled visual features.

Stage 3 — Slide-level: The visual backbone remains frozen. The slide-level transformer is trained on 155,604 high-resolution WSI images, aligning with TITAN and CARE to capture whole-slide spatial and contextual representations.

Each stage treats the corresponding teacher ensemble as a knowledge source; distillation is sequential (agglomerative) rather than joint, preventing interference across modalities and scales. At inference, task-appropriate modules are activated: vision-only tasks use the vision-only transformer; multimodal tasks additionally activate the multimodal transformer; slide-level tasks use the slide-level transformer on high-resolution inputs.

Key Contributions

  • Multi-stage agglomerative distillation framework that sequentially consolidates eight teacher PFMs (across three modality/scale categories) into one backbone without catastrophic forgetting across stages.
  • A comprehensive benchmark covering 21 task scenarios, 96 tasks, and 48 data sources across vision-only, vision-language, and slide-level settings — the broadest evaluation of any single PFM to date.
  • Demonstration that a single unified model can outperform task-matched specialists in all three evaluation regimes simultaneously.
  • Public release of model weights and code.

Results

Vision-only setting (vs. UNI-2, Virchow-2, H-Opt-1, GPFM): - Best average rank in linear-probe ROI classification (rank 1.25 across 12 tasks), KNN classification (1.58), image-to-image retrieval (1.33, MVAcc@5), and 8-shot ProtoNet few-shot classification (1.08, outperforming second-best by 1.50 rank positions). - Best overall rank in 7-task PMT segmentation despite being slightly below UNI-2 in instance segmentation AP@50 (0.599 vs. 0.607). - Best on 17/19 WSI ABMIL classification tasks; significantly better than second-best on 12. - Exceeds second-best model by 1.79 pp in average performance across the vision-only setting.

Vision-language multimodal setting (vs. MUSK, KEEP, CONCH): - Best average rank in ROI zero-shot classification (1.50), cross-modal retrieval (1.17, Recall@50), ROI VQA (1.87), WSI MI-Zero zero-shot (1.17), and 10-shot PathPT few-shot WSI (1.17). - Exceeds second-best model by 6.39 pp average in this setting. - Strong performance on 9 multimodal clinical WSI tasks (MICA) including vascular invasion detection and treatment response prediction; multimodal fusion consistently outperforms unimodal ablations.

Slide-level setting (vs. TITAN, CARE): - Best average rank in KNN WSI classification (1.53), linear-probe WSI classification (1.47), slide-to-slide retrieval (1.62, MVAcc@3), biomarker prediction (1.71, AUC), and survival analysis (1.67, C-index). - Exceeds second-best by 3.04 pp average; exceeds mean of remaining models by 4.00 pp.

Limitations

  • The paper text provided is truncated; slide-level few-shot results and fine-tuning results are cut off and cannot be fully assessed from the excerpt.
  • Distillation depends entirely on the quality and coverage of the eight teacher models; systematic biases or dataset gaps in any teacher are inherited.
  • The three-stage sequential approach requires frozen prior modules at each subsequent stage, preventing end-to-end joint optimization that might further improve cross-modal alignment.
  • High-resolution slide-level training is limited to 155,604 images, which is substantially smaller than the tile-level corpus; generalization of the slide-level module to rare cancer types may be constrained.
  • Computational cost of distilling eight teachers and the multi-stage pipeline is not quantified in the excerpt.
  • Evaluation is entirely on frozen or lightly adapted features; the behavior of ALICE under full fine-tuning across all three modules is not characterized.

Relevance to Vision-Language Models

ALICE directly extends the vision-language pretraining paradigm (contrastive/generative alignment between image and text) into a domain where VLM capabilities must coexist with dense visual discriminability and spatial slide-level reasoning — challenges not well-addressed by general-domain VLMs. The multimodal stage of ALICE treats CONCH, MUSK, and KEEP (themselves pathology VLMs) as teachers, demonstrating a distillation strategy for transferring vision-language alignment into a model that simultaneously preserves non-linguistic visual representations. The competitive results on zero-shot classification, cross-modal retrieval, and VQA (which are canonical VLM evaluation tasks) under a pathology-specific benchmark provide concrete evidence for how VLM capabilities scale and interact with domain-specific visual pretraining. For VLM researchers, ALICE illustrates that modality-staged distillation — decoupling the learning of language alignment from morphological feature learning — can be a principled alternative to joint multimodal pretraining when teacher models of different types are available.