Skip to content

LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

🕒 Published (v1): 2026-07-13 08:38 UTC · Source: Arxiv · link

Why this paper was selected

Language-guided distillation from pathology FMs; Won-Ki Jeong; reduces WSI inference cost

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LaGuadia is a multi-teacher knowledge distillation framework for computational pathology that uses clinical language (extracted from pathology reports) to adaptively weight teacher contributions per patch, rather than applying uniform aggregation. An 87M-parameter ViT-B student trained this way matches or outperforms billion-parameter Pathology Foundation Models (PFMs) such as GigaPath (1.13B) and UNI (303M) on WSI captioning, VQA, and slide-level classification.

Problem

Existing PFMs for Whole Slide Image (WSI) analysis are computationally prohibitive (300M–1.13B parameters). Knowledge distillation can compress them, but current multi-teacher KD methods apply uniform or coarse confidence-based weighting that ignores tissue heterogeneity: the optimal teacher varies across tissue regions, and visual features alone provide no reliable ground truth for identifying which teacher is most expert for a given patch.

Method

LaGuadia operates in three stages:

  1. Keyword Extraction. GPT-5-mini extracts visually observable clinical keywords from pathology reports, filtered by three constraints: observability on H&E-stained WSIs, identifiability at 20× magnification, and minimal redundancy. A global Keyword Bank is built for use as hard negatives.

  2. Vision-Language Alignment via Meta-Teacher. A shallow projection layer is appended to each frozen teacher backbone. MedSigLIP (a pathology-specific VLM) assigns pseudo-labels to patches via highest cosine similarity to the patient's keyword set \(\hat{K}_p\): $\(\hat{K}_p^* = \arg\max_{k \in \hat{K}_p} \cos(\Phi^V(x), \Phi^L(k))\)$ Contrastive learning (SigLIP loss) then aligns visual embeddings to these pseudo-labels using hard negatives selected by an average-similarity threshold.

  3. Language-Guided Adaptive KD. A consensus score aggregates similarity votes across all teachers to select a per-patch Pseudo-Target Keyword \(\tilde{K}_x\). Teacher weights \(\omega_i(x)\) are then computed via softmax over each teacher's semantic alignment to \(\tilde{K}_x\): $\(\omega_i(x) = \frac{\exp(\cos(\text{Proj}_i(T_i^V(x)), \Phi^L(\tilde{K}_x))/\tau)}{\sum_j \exp(\cos(\text{Proj}_j(T_j^V(x)), \Phi^L(\tilde{K}_x))/\tau)}\)$ The distillation loss is a weighted cosine dissimilarity sum: $\(L_{KD} = \sum_i \omega_i(x) \cdot (1 - \cos(\text{MLP}_i(\text{student}(x)), T_i^V(x)))\)$ The student is a ViT-B (86M) initialized from DINOv3, fine-tuned via LoRA. Teachers are frozen GigaPath, UNI, and Virchow2.

Key Contributions

  • Language-guided per-patch adaptive teacher weighting in multi-teacher KD, resolving teacher selection under tissue semantic heterogeneity.
  • Average-similarity-based negative sampling strategy for contrastive learning that enforces harder, clinically meaningful negatives.
  • Consensus-based Pseudo-Target Keyword selection to prevent knowledge conflicts across teacher ensemble.
  • Empirical demonstration that an 87M LoRA-tuned student matches or exceeds 303M–1.13B PFMs across generative and discriminative pathology tasks.

Results

WSI Captioning (PathText dataset): - LaGuadia OVR: 0.2461 vs. GigaPath 0.2452, UNI 0.2458, GPFM 0.2457 - Best METEOR (0.2896) and BLEU-4 (0.1031) on STAD; best BLEU-4 (0.1015) on THCA

WSI VQA (WSI-VQA / WSI-Bench): - Highest OVR: 0.4184 (vs. second-best Virchow2 0.4182) - Highest \(F_{actent}\) on BRCA (0.9268) and STAD (0.4737), indicating superior factual consistency

MIL Classification (TCGA-BRCA/STAD/THCA): - Highest OVR: 0.6538, outperforming all baselines - Best survival (THCA: 0.5843) and progression (THCA: 0.8446)

Ablation (THCA cohort): - Uniform KD hurts \(F_{actent}\) (0.5017 → 0.4988) and survival (0.5425 → 0.5180) vs. base - LaGuadia restores and exceeds baseline: \(F_{actent}\) 0.5036, survival 0.5843, progression 0.8446

Limitations

  • Uses class token-level distillation only; patch-level supervision is not explored and may capture finer morphological detail.
  • Distillation is performed once across all folds rather than per fold, introducing potential cross-fold leakage.
  • Keyword extraction relies on GPT-5-mini, introducing dependency on a proprietary external model not directly evaluated for extraction quality.
  • Evaluation is limited to TCGA cohorts (BRCA, STAD, THCA); generalization to other cancer types or staining protocols is unverified.

Relevance to Vision-Language Models

LaGuadia is directly relevant to VLM research in the medical domain: it exploits MedSigLIP (a pathology VLM) as a meta-teacher to bridge the gap between visual patch representations and clinical linguistic semantics, demonstrating how vision-language alignment can serve as a reliable supervision signal even when patch-level ground truth is unavailable. The framework illustrates a broader principle—clinical or domain-specific language can act as a semantic anchor for distilling and reweighting multiple large visual encoders, a strategy transferable to other VLM-driven knowledge transfer settings. For researchers tracking VLMs, this work shows that language-guided adaptive distillation can produce compact encoders that surpass much larger vision-only and vision-language foundation models on factual consistency metrics like \(F_{actent}\).