Weight Group-wise Post-Training Quantization for Medical Foundation Model

🕒 Published (v1): 2026-04-09 00:34 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Permutation-COMQ is a post-training quantization (PTQ) method for medical foundation models (specifically MedSAM) that reorders weight matrix elements by magnitude before coordinate-wise minimization, then restores original ordering via inverse permutation. This eliminates the outlier-dominated scaling problem that degrades standard PTQ at aggressive bit-widths. It requires no backpropagation, no Hessian estimation, and no hyperparameter tuning.

Problem

Large medical foundation models like MedSAM (ViT-B backbone, 1.5M+ training masks) are too computationally expensive for terminal clinical devices requiring low-latency inference (e.g., surgical navigation, bedside diagnostics). Existing PTQ methods (RTN, GPTQ, COMQ) fail at low bit-widths because per-channel scale factors are dominated by weight outliers, causing coarse quantization resolution for the majority of small-magnitude weights and significant accuracy degradation.

Method

Permutation-COMQ extends COMQ (coordinate-wise minimization quantization) with a weight-aware permutation step:

1. Permute: For each linear layer's weight matrix W ∈ ℝ^(m×n), apply a permutation P that sorts elements in ascending magnitude order, producing Wp = PW. This clusters similar-magnitude weights into the same quantization units, reducing intra-unit dynamic range.
2. Quantize: Apply per-channel coordinate-wise minimization on Wp. Each quantized weight and scale factor is treated as a separate optimization coordinate; each univariate subproblem has a closed-form solution (dot products + rounding, no backprop). Scale factors δⱼ are updated via least-squares after each row sweep.
3. Reverse permute: Apply P⁻¹ to restore original weight ordering: W̃q = P⁻¹Wq.

The quantization granularity is per-channel (one scale factor per output channel column), which the ablation shows is critical—per-layer scaling at 4-bit degrades NSD to 49.4% vs. 93.1% for per-channel.

Key Contributions

• Permutation-COMQ: A PTQ algorithm that reorders weights by magnitude before coordinate descent quantization, reducing intra-unit magnitude heterogeneity and improving scale factor allocation for small weights.
• Backpropagation-free optimization: Reduces to closed-form univariate quadratic solutions using only dot products and rounding; no Hessian inverse, no hyperparameter tuning.
• Per-channel weight-aware strategy: Assigns independent scale factors per output channel column within the permuted framework, preserving channel structure after inverse permutation.
• SOTA results on MedSAM at 2/4/8-bit: Demonstrated on AbdomenCT-1K across liver, kidney, spleen, and pancreas segmentation.

Results

Evaluated on AbdomenCT-1K (1000+ CT scans, 12 centers) using MedSAM with ViT-B encoder. Metrics: DSC and NSD. Baselines: RTN, COMQ, full-precision (32-bit).

• 8-bit: DSC 93.615% / NSD 93.204% vs. COMQ (93.486% / 92.938%) and RTN (93.499% / 92.936%); near-lossless vs. FP32 baseline (93.505% / 92.969%)
• 4-bit: DSC 93.434% / NSD 93.089% vs. COMQ (91.874% / 90.011%) and RTN (90.526% / 86.755%); exceeds FP32 baseline on DSC
• 2-bit: DSC 86.939% / NSD 78.935% vs. COMQ (71.8% / 53.733%) and RTN (29.79% / 30.221%); largest gain margin
• Ablation (per-layer vs. per-channel at 4-bit): Per-layer achieves DSC 55.439% / NSD 43.685% vs. per-channel 93.434% / 93.089%—confirming per-channel granularity is essential

Limitations

• Evaluated on a single model (MedSAM/ViT-B) and a single dataset (AbdomenCT-1K); generalizability to other medical foundation models (e.g., LLM-based, multimodal) or imaging modalities (MRI, pathology) is undemonstrated.
• Only weight quantization is addressed; activation quantization is not considered, limiting achievable inference speedup on hardware without weight-only acceleration paths.
• No wall-clock inference latency or memory footprint measurements reported; compression benefits are argued theoretically but not empirically benchmarked on target terminal devices.
• Synthetic outlier simulation experiments do not fully capture real-world weight distribution diversity across different foundation model architectures or training regimes.
• The permutation step adds overhead to the PTQ process itself, though this is a one-time cost; no quantification of calibration time is provided.

Relevance to Foundation Models in Medicine

This paper directly addresses the deployment gap for medical foundation models—specifically the compression of MedSAM for resource-constrained clinical hardware—which is a critical bottleneck for real-world adoption of large medical AI. The approach is architecture-agnostic within the PTQ framework and could in principle extend to other ViT-based or transformer medical FMs beyond SAM. For those tracking medical FM efficiency, this establishes that aggressive 2-bit quantization of MedSAM-class models is viable with acceptable segmentation fidelity, opening a path toward on-device inference. It complements the growing body of work on medical FM adaptation (MedSAM, Med2D, SAM adapters) by addressing the orthogonal compression axis.