Skip to content

MAPLE: Multi-scale Attribute-enhanced Prompt Learning for Few-shot Whole Slide Image Classification

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MAPLE is a hierarchical prompt learning framework that adapts vision-language models to few-shot whole slide image (WSI) classification by explicitly modeling multi-scale histological entities (nuclei, cytoplasm, glands) and their subtype-specific phenotypic attributes, rather than relying solely on slide-level feature alignment. It combines LLM-generated entity-level and slide-level prompts with a cross-scale entity graph to produce both fine-grained and global predictions. MAPLE consistently outperforms existing MIL and prompt-based baselines across three TCGA cancer cohorts under 4/8/16-shot settings.

Problem

Existing VLM-based prompt learning methods for few-shot WSI classification (TOP, ViLa-MIL, MSCPT, FOCUS) aggregate patch features into a slide-level representation and align it with simple class-name or global-description prompts. This ignores the subtype-specific phenotypic variation of histological entities (e.g., nucleoli are small and inconspicuous in LUAD but large and prominent in LUSC) and discards the discriminative information present at multiple resolution scales, limiting both accuracy and interpretability.

Method

MAPLE processes each WSI at two resolution scales (5× low, 10× high) using a frozen PLIP image encoder. The pipeline has four stages:

  1. LLM-powered prompt construction: GPT-4 is queried to produce (a) generic visual descriptions and (b) subtype-specific attribute strings for each histological entity at each scale, plus slide-level summary prompts that reference entity names.
  2. Language-guided instance selection: Region-description text embeddings (from PLIP's text encoder) are cosine-compared against patch features; top-k% patches per scale are retained as tumor-related instances.
  3. Entity-guided cross-attention (EGCA): For each entity, its generic text embedding acts as a query over selected patch features (key/value) to extract an entity visual representation; cosine similarity against subtype-specific attribute embeddings yields entity-level classification logits.
  4. Cross-scale entity graph + slide-level aggregation: A GAT over all entity nodes (within and across scales), using top-k cosine-similarity neighbors, refines entity representations. A gated attention mechanism aggregates refined entities into scale-specific slide representations, which are aligned with slide-level prompts for slide-level logits. Final prediction is a weighted sum: λ · slide_logits + (1−λ) · mean(entity_logits).

All text prompts include prepended learnable context vectors (CoOp-style); only these vectors and the cross-attention/graph/aggregation modules are trained via cross-entropy; PLIP encoders are frozen.

Key Contributions

  • Hierarchical framework that performs explicit entity-level classification via cross-attention between patch features and LLM-generated entity descriptions, rather than treating entities implicitly through aggregation.
  • Cross-scale entity graph (GAT) that propagates information between semantically related entities at low and high resolution, enriching entity representations with inter-scale context.
  • LLM-generated multi-scale prompts: subtype-specific phenotypic attribute strings for each entity (e.g., "nucleolus: frequently small, inconspicuous" for LUAD vs. "large, prominent" for LUSC) replace generic class-name templates.
  • Language-guided instance selection to filter tumor-relevant patches before entity extraction.
  • Complementary combination of entity-level and slide-level logits with a tunable mixing weight λ.

Results

All results are 5-fold cross-validation mean ± std; baselines include ABMIL, TransMIL, GTMIL, WiKG, TOP, ViLa-MIL, MSCPT, FOCUS; backbone is PLIP for all.

  • TCGA-BRCA (2-class): 16-shot AUC 80.1% vs. ViLa-MIL 78.9% (+1.2%); 4-shot AUC 72.2% vs. FOCUS 70.3% (+1.9%).
  • TCGA-RCC (3-class): 16-shot AUC 96.9% vs. FOCUS 95.1% (+1.8%); 8-shot AUC 95.7%; 4-shot AUC 90.9%.
  • TCGA-NSCLC (2-class): 16-shot AUC 90.3% vs. FOCUS 86.2% (+4.1%); 8-shot AUC 85.5% vs. FOCUS 81.8% (+3.7%); 4-shot AUC 74.0% vs. FOCUS 71.3%.
  • Ablation (16-shot): removing language-guided selection, EGCA, or cross-scale graph each degrades all three datasets; entity-only and slide-only variants are both inferior to the combined MAPLE.
  • Multi-scale ablation: combining both scales outperforms using either high-resolution-only or low-resolution-only entities.

Limitations

  • Relies on GPT-4 (closed, costly) to generate entity and slide-level prompts; prompt quality is LLM-dependent and not validated across other LLMs in the main paper (only an appendix comparison).
  • Experiments use only PLIP as the VLM backbone in the main results; generalizability to other pathology VLMs (CLIP, CONCH) is deferred to appendices.
  • Hyperparameter sensitivity: four key hyperparameters (entity count nk, graph neighbors ne, patch selection ratio r, mixing weight λ) all require tuning; few-shot settings make reliable validation splits small.
  • Entity set is fixed by LLM query at construction time; the framework does not adapt entity selection based on the specific slides or the few-shot support set at inference.
  • Evaluated only on TCGA data; performance on out-of-distribution cohorts (different scanners, staining protocols, institutions) is untested.

Relevance to Vision-Language Models

MAPLE is a direct application of prompt learning—a technique developed in the VLM literature (CoOp, CoCoOp)—to the domain of computational pathology, extending it from single-image to gigapixel multi-instance settings. The key VLM-specific insight is that CLIP-style contrastive alignment can be made more discriminative by grounding prompts in domain-specific ontologies (histological entities) rather than generic class names, an approach relevant to any VLM adaptation task where intra-class visual variation is structured and describable. The use of LLMs to generate rich, subtype-specific attribute strings as prompt text demonstrates a general strategy for injecting expert knowledge into VLM fine-tuning without annotated data, which is broadly applicable to medical and scientific domains beyond pathology. The combination with multi-scale features and cross-attention entity extraction extends the VLM+MIL paradigm and sets a stronger baseline for future few-shot medical image understanding work.