Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identification¶

🕒 Published (v1): 2025-07-25 06:59 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

DCM-DeID is a divide-and-conquer framework for medical image de-identification that separates identity removal from medical semantic preservation. It uses pre-trained Medical Foundation Models (MFMs) to compensate lost diagnostically relevant information, then applies Minimum Description Length (MDL)-based feature decoupling to remove residual identity cues from MFM features before diffusion-based image re-synthesis. Evaluated across seven datasets and three task types, it achieves state-of-the-art utility–privacy trade-offs.

Problem¶

Existing medical image de-identification (DeID) methods fail on two axes: (1) they cannot preserve task-generalizable medical semantics across downstream tasks (classification, segmentation, captioning) because identity and diagnostic features are entangled in raw image space; and (2) they do not support flexible, adjustable privacy levels — methods that work at low attack success rates degrade catastrophically at high privacy settings. Joint optimization of privacy and utility objectives proves suboptimal in practice.

Method¶

DCM-DeID operates in three sequential stages:

ID-Blocking: A ViT-based ReID network extracts patch-level features, computes cosine similarity of each patch to the global identity embedding, producing a spatial similarity map. Binarizing this map at threshold quantile r ∈ {0.95, 0.7, 0.4, 0.2} yields an adjustable ID mask; masked regions are zeroed in the image (X_noID).
ID-Free Medical Semantics Extraction: A frozen domain-specific MFM (MGCA-ResNet for X-ray, RetFound-ViT for fundus) extracts features f_MFM from the original image. A feature encoder quantizes these into a 32-channel discrete integer code Q under MDL regularization (rate-distortion loss with a GMM-based codelength estimator). A learned binary mask M (trained via Gumbel-Softmax) partitions Q into identity codes Q_id and semantic codes Q_sem = (1-M)⊗Q. Only Q_sem is decoded back to produce the identity-free semantic feature f̂_sem.
Image Re-Synthesis: A latent diffusion model (Stable Diffusion UNet backbone) receives dual conditioning: X_noID (privacy-level signal) processed via a VAE + convolution Feat-Net, and f̂_sem (semantic signal), fused via bi-directional cross-attention. The model synthesizes a realistic de-identified image usable directly in downstream tasks without adaptation.

The full pipeline is end-to-end trained with L_total = L_code-all + L_code-id + L_diffuse.

Key Contributions¶

First benchmark for adjustable medical DeID with fair reproduction of five baselines across seven datasets and three downstream task types.
DCM-DeID framework decoupling identity removal (ID-Blocking) from semantic recovery (MFM-based compensation) into separate, non-conflicting steps.
MDL-based feature decoupling strategy using discrete code quantization + GMM entropy modeling to isolate and discard identity-associated components in MFM feature space.
Demonstration that MFM features themselves contain substantial identity information (~44% for X-ray, ~55% for fundus by bits-per-pixel analysis), necessitating explicit decoupling.

Results¶

At CMC-R1 = 10% attack success rate (hardest privacy setting):

X-ray classification (MIMIC-X): Ours 72.86% AUROC vs. best baseline Feat-Noise 65.84%; gap widens further at CMC-R1 = 5% where ID-Adv (56.32%) and Privacy-Net (54.21%) fall below even Pixel-Blur (61.62%).
X-ray classification (CheXpert): Ours 86.12% vs. MAE 77.05%, Pixel-Blur 74.45%.
X-ray captioning (MIMIC-X): Ours BLEU 0.1750 vs. Feat-Noise 0.1453, MAE 0.1186, Pixel-Blur 0.1259.
X-ray segmentation (ChestX-Det): Ours Dice 27.98% vs. ID-Adv 22.23%, Feat-Noise 10.74%, Pixel-Blur 7.53%.
Fundus classification (EyePACS): Ours 65.23% AUROC vs. second-best ID-Adv 56.28%.
Fundus segmentation (Refuge2): Ours Dice 54.66% vs. ID-Adv 41.23%, Privacy-Net 39.23%, MAE 27.33%.
Ablation: removing semantic compensation drops AUROC >8%; removing decoupling increases CMC-R1 by ~15% at equivalent AUROC (identity leaks back through raw MFM features).

Limitations¶

Inference latency is 540 ms (vs. 120–124 ms for non-diffusion baselines) due to multi-step diffusion; authors plan single-step diffusion integration.
Currently limited to 2D images (chest X-ray, fundus); MRI and multi-slice volumetric data are explicitly deferred to future work.
MFMs used (MGCA-ResNet, RetFound-ViT) are modality-specific; generalization to other modalities (CT, ultrasound, pathology) is unvalidated.
The MDL decoupling assumes a learned binary mask can cleanly separate identity from semantics in a shared code space; no theoretical guarantee that the partition is complete at all privacy levels.
All training is on MIMIC-X and EyePACS; cross-dataset generalization is evaluated but not trained for, which may limit performance on imaging systems with different acquisition characteristics.

Relevance to Foundation Models in Medicine¶

This paper demonstrates a concrete downstream application of medical foundation models (MGCA, RetFound) as semantic feature extractors in a privacy-preserving pipeline — a role distinct from the typical fine-tuning or zero-shot evaluation paradigm. The finding that MFM features contain ~44–55% identity-related information by codelength is an important negative result for the field: deploying MFMs in de-identification pipelines without explicit identity filtering reintroduces the very privacy risk they are meant to mitigate. The MDL-based decoupling approach suggests a general mechanism for separating unwanted attributes from rich foundation model representations, relevant to any setting requiring attribute-controlled feature extraction (e.g., domain shift, bias mitigation). For researchers tracking medical foundation models, this work highlights that the semantic richness of MFMs is both an asset and a liability in privacy-sensitive clinical deployments.