Skip to content

MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MoodAngels is a retrieval-augmented multi-agent framework for mood disorder diagnosis that decomposes psychiatric scale scoring to item level, retrieves DSM-5 criteria and anonymized case precedents, and synthesizes three specialized diagnostic agents via structured debate. It introduces MoodSyn, a 1,173-case synthetic dataset built on a VAE-diffusion pipeline for privacy-safe research. The base agent alone surpasses GPT-4o by 12.3% accuracy on 561 real clinical cases.

Problem

Psychiatric diagnosis lacks objective biomarkers, depends on subjective self-reported and clinician-estimated scales, and suffers from symptom overlap across mood disorders and other conditions. Clinical data is privacy-sensitive and scarce, limiting AI training and evaluation. Existing LLM-based medical agents rely on concrete lab results unavailable in psychiatry, and aggregate scale scores discard intra-scale symptom patterns critical for atypical presentations.

Method

MoodAngels operates in three coordinated layers:

Granular-Scale Analysis. Pearson correlation between each scale item and mood disorder presence is computed across the full dataset; the top 5% highest-correlation questions (plus clinically mandated PHQ-9 items) are selected and grouped into five symptom clusters (depression, suicidal ideation, energy/interest loss, anxiety, insomnia). Numeric scores are converted to textual descriptions and reorganized into coherent narrative paragraphs for LLM consumption. Within-group response consistency is used to flag discrepancies (e.g., high self-report depression score vs. low clinician-observed score) and trigger additional behavioral marker validation.

Retrieval Datastore. DSM-5 diagnostic and differential criteria are structured into a dense-vector knowledge base. An anonymized corpus of 2,243 real hospital cases is split 80/20 for retrieval/test. BGE-M3 is used for semantic embedding; top-5 similar criteria and/or cases are retrieved per query.

Three Diagnostic Agent Variants + Multi-Agent Debate. - Angel.R: no case retrieval — uses DSM-5 criteria + granular scale output only. - Angel.D: retrieves top-5 similar cases and presents them as raw context. - Angel.C: retrieves top-5 similar cases and returns a comparative analysis summary.

All three produce independent diagnoses. A Judge Agent checks for consensus; on disagreement, a Positive Debate Agent (pro mood-disorder) and Negative Debate Agent argue iteratively, with the Judge terminating debate and issuing a final verdict.

MoodSyn Dataset. Synthetic cases are generated via TabSyn, which uses a VAE to learn latent clinical feature distributions and a diffusion model for conditional denoising. Dynamic loss weighting enforces consistency between item-level responses and aggregate scale totals.

Key Contributions

  • First multi-agent framework targeting mood disorder diagnosis specifically, with item-level granular scale decomposition and structured debate-based synthesis.
  • Three agent variants (Angel.R/D/C) that span a spectrum from zero historical-case reliance to comparative case analysis, mitigating overreliance on prior cases.
  • MoodSyn: open-source 1,173-case synthetic psychiatric dataset with 25 features per case, validated for statistical fidelity, ML utility, and privacy via near-indistinguishability detection tests.
  • Ablation confirming that narrative-style medical records outperform structured formats for atypical presentations, and that scale item selection is critical to performance.

Results

  • On 561 real clinical cases: multi-Angels (GPT-4o backbone) achieves ACC = 0.925, Recall = 0.881, Macro F1 = 0.917, MCC = 0.832.
  • Angel.R (GPT-4o) reaches ACC = 0.920 vs. GPT-4o baseline ACC = 0.797 — a +12.3% accuracy gain.
  • DeepSeek-V3 backbone: multi-Angels ACC = 0.923, Recall = 0.866, Macro F1 = 0.916.
  • All MoodAngels variants outperform all five baselines (LLaMA3-8B, Mistral-7B, GPT-4o, DeepSeek-V3, medgemma-27b) on both real and synthetic data.
  • Angel.D outperforms Angel.R, but Angel.C slightly underperforms Angel.D, indicating diminishing returns from comparative case analysis.
  • On MoodSyn (140 test cases): multi-Angels (DeepSeek-V3) ACC = 0.821, Macro F1 = 0.821.

Limitations

  • Evaluated on a single hospital's proprietary dataset (561 real cases); generalizability to other clinical settings is unvalidated.
  • Angel.C's performance dip suggests over-reliance risk when comparative case analysis is applied — the optimal retrieval strategy is not fully characterized.
  • MoodSyn covers only the 25 features used in this study; it cannot substitute for full-record clinical datasets in broader psychiatric tasks.
  • The debate mechanism's convergence behavior and number of rounds needed are not fully ablated.
  • Narrative medical records are required for best performance, but their availability in real deployments varies and their processing pipeline is complex.
  • No evaluation on disorders beyond the binary mood-disorder/non-mood-disorder split (e.g., fine-grained bipolar vs. depression differentiation).

Relevance to Harnesses / Meta-Harnesses

MoodAngels is a domain-specific multi-agent harness with a fixed orchestration topology: three specialized sub-agents with differentiated tool access (retrieval on/off, display vs. comparative), a Judge meta-agent, and conditionally spawned Debate Agents — a clear example of a structured agentic pipeline where the harness governs control flow, tool routing, and agent instantiation policy. The pattern of running parallel independent agents with heterogeneous retrieval strategies and merging via debate is directly analogous to meta-harness designs that fan out to specialist sub-agents and aggregate results. The MoodSyn construction pipeline (TabSyn + diffusion) also illustrates how domain harnesses must address data-availability constraints not just at inference time but at the harness-training and evaluation level, a concern generalizable to any harness operating in low-data, privacy-constrained domains.