Skip to content

Embeddings to Diagnosis: Latent Fragility under Agentic Perturbations in Clinical LLMs

🕒 Published (v1): 2025-07-27 16:48 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Clinical LLMs achieve high benchmark scores yet exhibit severe latent instability when inputs are perturbed in clinically realistic ways (e.g., masking a symptom, negating a finding). The authors introduce LAPD, a geometry-aware evaluation framework, and LDFR (Latent Diagnosis Flip Rate), a metric that detects diagnostic boundary crossings in PCA-reduced embedding space that surface metrics like BERTScore completely miss. Results confirm this fragility across both synthetic and real MIMIC-IV clinical notes.

Problem

Standard NLP metrics (ROUGE-L, BERTScore, NER-F1) are insensitive to latent representation shifts that cause diagnosis changes in clinical LLMs. Models can maintain BERTScore > 0.89 at 25% entity masking while their diagnosis flips—a gap that existing robustness frameworks (MedFuzz, PerturbScore) do not capture because they evaluate output labels, not internal representational stability.

Method

LAPD operates in three stages:

  1. Agentic note generation: Clinical notes are synthesized from DDXPlus via a three-step chained prompting pipeline (forward symptom-to-diagnosis reasoning → backward justification → narrative construction), producing 100 notes per diagnosis across 7 conditions.
  2. Structured perturbation: Four perturbation axes are applied at 0–100% intensity in 25% increments: entity masking ([MASK] replacement), negation (polarity reversal), synonym substitution (clinical term swaps), and numerical distortion (±5–15% on vitals/labs). This yields 2,000 synthetic and 1,800 real (DiReCT/MIMIC-IV) perturbed samples.
  3. Latent probing: Notes are encoded with frozen ClinicalBERT (synthetic) or Clinical-Longformer (real notes). PCA retaining 90% variance yields 30–40 components. A logistic regression classifier trained on unperturbed embeddings defines latent decision boundaries. LDFR = fraction of perturbed samples whose PCA-projected embedding crosses that boundary, formally: LDFR(t) = (1/N) Σ 𝟙[d₀⁽ⁱ⁾ ≠ d_t⁽ⁱ⁾]. Complementary metrics include centroid displacement and per-dimension PCA variance redistribution.

Key Contributions

  • LAPD framework: end-to-end geometry-aware evaluation pipeline combining agentic note generation, structured clinical perturbation, and PCA-based latent probing.
  • LDFR metric: model-agnostic signal quantifying diagnostic boundary crossings in PCA-reduced latent space, capturing instability invisible to surface metrics.
  • Empirical finding: entity masking is the most destabilizing perturbation (LDFR drops from 0.9125 to 0.3525 at full masking in synthetic notes; from 0.6923 to 0.4198 on DiReCT), while numerical perturbations are least disruptive.
  • Generalization validation: LDFR patterns replicate on 90 real discharge summaries from MIMIC-IV DiReCT, confirming the metric works beyond synthetic settings.
  • Dimensional collapse finding: masking causes up to 20–30% of total embedding variance to concentrate in a single PCA dimension (bottleneck effect), while centroid drift is a poor predictor of diagnosis flips.

Results

  • At 25% entity masking: BERTScore remains > 0.89, yet LDFR drops from 0.9125 → 0.5525 (synthetic) and Pearson correlation with LLM predictions drops from 0.944 → 0.493.
  • At 100% masking: LDFR = 0.3525 (synthetic), Pearson = 0.317; indicating near-complete breakdown of latent alignment.
  • Negation: LDFR degrades from 0.9125 → 0.7050 at full perturbation; smaller than masking but consistent.
  • Synonym replacement: LDFR 0.9125 → 0.7325; numerical: largely stable (0.9125 → 0.8925).
  • On DiReCT real notes: masking LDFR 0.6923 → 0.4198; negation causes sharper degradation than in synthetic notes.
  • Model comparison: GPT-4o and GPT-3.5-turbo show lower LDFR under synonym/numeric edits; LLaMA-3.1 and Mistral-7B more fragile under masking/negation, with LDFR > 0.5 in some cases; Mistral-7B worst overall.
  • Clinician evaluation of synthetic notes: structure/reasoning scores 2.0–2.7 out of 3.

Limitations

  • Synthetic notes lack real-world clinical variability; clinician reviewers noted missing vitals, incomplete differentials, and unrealistic symptom-diagnosis pairings.
  • Latent boundaries are defined using ClinicalBERT/Longformer embeddings, which may not align with the internal representations of the generative LLMs being probed.
  • No expert adjudication of whether perturbed inputs should clinically warrant a diagnosis change—LDFR cannot distinguish appropriate vs. inappropriate flips.
  • PCA + logistic regression assumes linear decision boundaries; non-linear manifold structure is unaddressed.
  • Small real-note sample (90 notes); clinical note generation evaluated on only 5 notes by 2 clinicians.
  • MedGemma-27b is listed as evaluated but model-specific LDFR numbers are not broken out in the tables presented.

Relevance to Foundation Models in Medicine

This paper directly addresses a critical safety gap for foundation models deployed in clinical settings: high benchmark accuracy does not imply robustness under the routine input variability of real clinical documentation (redaction, negation, synonym use). The LDFR metric provides a model-agnostic auditing tool applicable to any clinical foundation model—whether general-purpose (GPT-4o, LLaMA) or medically fine-tuned (MedGemma)—without requiring ground-truth labels for perturbed inputs. For researchers tracking medical FM evaluation and deployment safety, this work complements output-level robustness studies (MedFuzz) with a geometry-aware latent-space lens that may become a standard pre-deployment audit component.