Skip to content

LapFM: A Laparoscopic Segmentation Foundation Model via Hierarchical Concept Evolving Pre-training

🕒 Published (v1): 2025-12-09 10:09 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LapFM is a laparoscopic surgical segmentation foundation model that learns from massive unlabeled surgical images via a Hierarchical Concept Evolving Pre-training paradigm. It constructs a unified taxonomy (Anatomy, Tissue, Instrument) and iteratively self-labels unlabeled data using confidence-filtered pseudo-labels, producing LapBench-114K. It substantially outperforms SAM-family surgical adapters across anatomy, tissue, and instrument segmentation without requiring manual prompts.

Problem

Surgical segmentation suffers from annotation scarcity, semantic inconsistency across datasets (varying granularities and category spaces), and over-reliance on SAM fine-tuning via limited supervised data. Existing surgical SAM variants treat foundation models as domain adapters with fixed category spaces and thus fail to generalize across annotation granularities or unseen surgical categories. Generic SSL pre-training (masked autoencoders, contrastive learning) learns coarse global representations unsuitable for pixel-precise surgical parsing.

Method

LapFM combines two components under a Hierarchical Concept Evolving Pre-training framework:

Laparoscopic Concept Hierarchy (LCH): A three-branch taxonomy (Anatomy, Tissue, Instrument) with parent-child node structure. A hierarchical mask decoder with separate parent and child query embeddings enforces cross-granularity semantic consistency. Parent-level queries attend to image embeddings to produce coarse masks; a parent-specific feature enhancer then conditions child queries via cross-attention on parent masks (Eq. 5–7). Hierarchical consistency is enforced via a KL-divergence loss (Eq. 8) between parent predictions and the aggregation of their children's predictions, alongside standard BCE + Dice losses.

Confidence-driven Evolving Labeling: Starting from a SAM2 ViT-H backbone with adapters in FFN layers, LapFM is trained on five fully-annotated datasets. It then generates pseudo-labels on unlabeled surgical images (CholecT50, Cholec80, EndoScapes-CVS201) and ranks masks by predicted Dice confidence scores (small confidence token heads trained with MSE loss, Eq. 9). The top 70% confidence pseudo-labels are added to training; the remaining 30% are retained as low-confidence seeds. Three iterations of this process build LapBench-114K (114K image-mask pairs, 20 surgical categories).

Key Contributions

  • LapFM: a prompt-free laparoscopic segmentation foundation model trained on unlabeled surgical video data via hierarchical evolving pre-training.
  • Laparoscopic Concept Hierarchy (LCH) unifying Anatomy/Tissue/Instrument into a scalable, searchable taxonomy with explicit parent-child conditional probabilities.
  • Confidence-driven Evolving Labeling: iterative pseudo-labeling engine with Dice-score confidence tokens for automatic quality filtering.
  • LapBench-114K: 114K image-mask pairs across 20 surgical categories from 8 public datasets, validated by a certified surgeon (80.07% average pseudo-label Dice vs. expert annotation).

Results

  • Anatomy segmentation (12 categories): LapFM achieves 78.1% average Dice vs. ReSurgSAM2 68.5% (+9.6% absolute); average HD 186mm vs. 274mm for ReSurgSAM2 (−32.1%); best in 10/12 categories including Pancreas 46.7% (vs. best baseline 13.7%, 3.4× improvement).
  • Tissue segmentation: LapFM 94.8% average Dice vs. ReSurgSAM2 73.1% (+21.7% absolute); Blood category (C12) 98.4% Dice vs. 48.8% for ReSurgSAM2; HD 9.7mm vs. 80.4mm (7.3× reduction).
  • Instrument segmentation (8 categories): LapFM 94.6% average Dice vs. best baseline MedSAM 86.3%; Specimen Bag (C18) 97.6% vs. 62.8% best baseline (+35.0% absolute).
  • Unseen generalization (GynSurg dataset, 6 categories): LapFM 52.95% average Dice vs. ReSurgSAM2 46.36% (+8.25%), Zig-RiR 20.17% (+34.45%); p < 0.001 over all surgical SAM variants.
  • Ablation (Table IV): Full LapFM averages 89.97% Dice vs. SAM2 baseline 80.84% (+9.13%); LCH alone contributes +1.67% Dice; Confidence-driven Evolving Labeling with full architecture contributes largest gains.

Limitations

  • LapBench-114K pseudo-labels have an average Dice of only 80.07% vs. surgeon annotations, indicating non-trivial noise that may affect rare/fine-grained categories.
  • Evaluated exclusively on laparoscopic cholecystectomy and hysterectomy procedures; generalization to other surgical specialties (cardiac, orthopedic, neurosurgery) is unvalidated.
  • The hierarchical taxonomy (LCH) is manually designed with three fixed top-level branches; scalability to modalities beyond laparoscopy is unclear.
  • HD remains high for some anatomy categories (>270mm) even for LapFM, suggesting boundary precision degrades on fine-grained structures.
  • Requires two NVIDIA A100 GPUs and three full evolving iterations; computational cost for new procedure domains is non-trivial.
  • Prompt-free mode trades recall on rare categories for autonomy; the paper does not analyze false-positive rates across categories.

Relevance to Foundation Models in Medicine

LapFM exemplifies a domain-specific surgical foundation model that moves beyond SAM adaptation by building its own pre-training paradigm and data engine from surgical-domain unlabeled images—directly addressing the annotation bottleneck central to medical AI. The Confidence-driven Evolving Labeling is a practically important design pattern for any medical domain where labeled data is scarce but video or image data is abundant. The hierarchical semantic taxonomy approach connects to broader work on universal segmentation models (e.g., UniverSeg, SAM) and demonstrates that surgical-specific ontological structure is necessary for cross-granularity generalization—a finding relevant to other medical imaging tasks (radiology, pathology) where category systems vary across institutions and datasets. This work establishes a benchmark (LapBench-114K) and baseline that will likely anchor future surgical foundation model comparisons.