Skip to content

Layer-wise Cross-Lingual Depression Detection from Speech: Analysis with Contrastive Alignment

🕒 Published (v1): 2026-07-03 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CLeaD is a supervised contrastive alignment framework that maps frozen WavLM speech embeddings from English and Mandarin into a shared clinical label space for cross-lingual depression detection. The paper corrects a widespread evaluation artifact — speaker-identity leakage from segment-level random splits — which artificially inflated prior Mandarin F1 scores to 0.954. Under rigorous leave-one-speaker-out evaluation, CLeaD modestly improves depressed-class speaker recall over baselines (F1: 0.640 vs. 0.622 on LOSO) and reveals that model scaling degrades cross-lingual transfer while improving monolingual performance.

Problem

Cross-lingual generalization of speech-based depression detection is an open problem. Prior work (1) is English-centric and transfers poorly to tonal languages like Mandarin, where lexical pitch contours mask prosodic depression markers, and (2) uses segment-level random splits without speaker grouping, causing speaker-identity leakage that inflates reported metrics. The paper quantifies that leakage alone causes a +0.23 F1 jump (0.628 → 0.856) on Mandarin, making prior cross-lingual benchmarks unreliable.

Method

CLeaD is a two-head network trained on frozen WavLM embeddings:

  1. Feature extraction. WavLM-Base-Plus (12 layers, \(d{=}768\)) or WavLM-Large (24 layers, \(d{=}1024\)) is used as a frozen extractor. Mean-pooled utterance embeddings at layer \(l\) are computed as \(\bar{h}^{(l)} = \frac{1}{F}\sum_{f=1}^{F} h_f^{(l)} \in \mathbb{R}^d\).

  2. Projection head. Maps \(h\) to an \(\ell_2\)-normalized 128-dim space: \(z = \ell_2\text{-norm}(W_2 \,\text{ReLU}(\text{BN}(W_1 h)))\), with dropout 0.3.

  3. Classification head. Two-layer MLP (\(\mathbb{R}^{128} \to \mathbb{R}^{64} \to \mathbb{R}^2\)).

  4. Training objective. Combined supervised contrastive (SupCon) and class-weighted cross-entropy: $\(\mathcal{L} = \lambda \mathcal{L}_{\text{SupCon}} + (1-\lambda)\mathcal{L}_{\text{CE}}, \quad \lambda = 0.5\)$ where same-label cross-lingual pairs (English depressed, Mandarin depressed) form positives in the SupCon loss with temperature \(\tau=0.1\).

Mixed-language batches are required for CLeaD to engage (cross-lingual positive pairs must co-occur in each batch). English segments are subsampled to match MODMA segment count. Speaker-independent splits with participant-ID grouping prevent leakage. Evaluation uses leave-one-speaker-out (LOSO) on all 52 MODMA speakers as the primary Mandarin metric, plus a 10-speaker held-out set for Dep-Rec (depressed-class speaker recall).

Key Contributions

  • Introduces CLeaD, a supervised contrastive framework for cross-lingual clinical speech alignment without parallel data or target-language fine-tuning.
  • Exposes and quantifies speaker-identity leakage: segment-level splits alone inflate Mandarin F1 by 0.23 (0.628 → 0.856); feature-scaling leakage contributes <0.001. Prior state-of-the-art Mandarin F1 of 0.954 is reproduced and shown to be an artifact.
  • Layer-wise analysis of WavLM-Base-Plus (layers 6–9) and WavLM-Large (layers 12–18) across six transfer conditions, validated with 95% bootstrap CIs (2000 resamples).
  • Demonstrates that model scale degrades cross-lingual performance: Large outperforms Base-Plus monolingually (EN AUC [0.825, 0.846] vs. [0.763, 0.787]) but consistently underperforms cross-lingually at every layer pair (non-overlapping CIs).
  • t-SNE and quantitative metrics confirm CLeaD reduces language separability (language LR accuracy: 89.2% → 84.0%) while increasing clinical cluster cohesion (clinical silhouette: 0.010 → 0.066).

Results

  • LOSO primary metric (52 MODMA speakers, MIX→ZH, Base-Plus L7): CLeaD Spk F1 = 0.640 vs. CLeaD w/o SupCon = 0.622; SVM-Linear leads at 0.762, LR at 0.714.
  • Dep-Rec (5 depressed speakers, 10-speaker held-out, MIX→ZH): CLeaD achieves 4/5 at Layers 7–8, vs. CLeaD w/o SupCon at 1/5 (L7) and 3/5 (L8); SVM-Linear achieves 2/5–3/5.
  • Peak segment F1 (MIX→ZH, Base-Plus): CLeaD = 0.561 at Layer 8.
  • Scale reversal confirmed: Large MIX→ZH CLeaD Spk F1 = 0.691 vs. Base-Plus 0.640, but Large segment F1 collapses across all cross-lingual conditions (e.g., MIX→ZH CLeaD: Base-Plus CI [0.462, 0.513] vs. Large [0.184, 0.236]).
  • Leakage ablation (Base-Plus L6, LR): Airtight MODMA F1 = 0.628/AUC 0.706; with speaker leakage = 0.856/0.933.
  • EN→ZH zero-shot: CLeaD = 1/5 Dep-Rec; GRU achieves 3/5 but collapses (predicts all as depressed in Large).
  • ZH→EN: CLeaD achieves highest F1 = 0.505 among all models.

Limitations

  • Small test set (\(n{=}52\) MODMA speakers, 5 depressed in held-out set); each Dep-Rec change = 20 percentage points, making held-out Dep-Rec illustrative rather than statistically definitive.
  • CLeaD requires cross-lingual positive pairs in each batch; it structurally cannot provide alignment in zero-shot (EN→ZH) settings — SupCon gradient is identically zero without Mandarin in training.
  • WavLM is pretrained on 94K hours of English-only speech; the English-only pretraining ceiling limits cross-lingual headroom independently of the contrastive objective.
  • Baselines received no hyperparameter search, potentially understating their ceilings.
  • Demographic confounds between E-DAIC and MODMA (age, recording conditions, interview structure) are not disentangled from language effects.
  • CLeaD does not achieve state-of-the-art Spk F1; SVM-Linear and LR outperform it on the LOSO primary metric.

Relevance to Vision-Language Models

CLeaD's use of supervised contrastive alignment to bridge two domain distributions (English/Mandarin speech) into a shared semantic space is architecturally analogous to CLIP-style cross-modal alignment in VLMs, where image and text encoders are aligned via contrastive objectives on paired positives. The layer-wise finding — that intermediate layers of large SSL models carry the most transferable, modality/language-agnostic representations — directly parallels observations in vision-language research that intermediate ViT or BERT layers encode more universal features than final layers specialized for pretraining objectives. The paper's rigorous exposure of evaluation leakage is a methodological cautionary tale relevant to any multimodal or cross-domain benchmark, including VLM evaluations that risk identity or spurious correlation artifacts. Finally, the finding that model scale hurts cross-lingual transfer (while helping monolingually) echoes debates in VLM scaling about whether larger models improve cross-domain compositional generalization.