Skip to content

MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MoodAngels is a retrieval-augmented multi-agent framework for mood disorder diagnosis that decomposes psychiatric scale scores into item-level granular analysis and synthesizes independent diagnoses via structured debate. It outperforms GPT-4o by 12.3% accuracy on real clinical cases. The authors also release MoodSyn, a synthetic dataset of 1,173 privacy-preserving psychiatric cases.

Problem

Psychiatric diagnosis—specifically mood disorders (depression, bipolar)—is poorly served by general-purpose LLM agents because: (1) it relies on subjective self-reported and clinician-estimated scales rather than objective biomarkers; (2) symptoms overlap extensively across disorders; (3) sensitive patient data cannot be publicly shared, starving AI research of training and evaluation data; and (4) existing LLMs aggregate scale total scores, losing fine-grained symptom information.

Method

MoodAngels combines three interacting components:

  1. Granular-Scale Analysis: Rather than using aggregate scale totals, Pearson correlation is computed between each item in 13 psychiatric scales and mood disorder presence; the top 5% most correlated items are selected and grouped into five symptom clusters (depressive mood, loss of interest, anxiety, insomnia, suicidal ideation). Numeric scores are converted to textual descriptions and reformatted as narrative paragraphs.

  2. Retrieval Datastore: DSM-5 diagnostic and differential criteria are structured into a vector knowledge base. Anonymized real-world cases (2,804 total, 80% used for retrieval) are stored alongside. BGE-M3 dense embeddings retrieve the top-5 similar criteria or cases at inference time.

  3. Three Diagnostic Agents + Multi-Agent Debate: Three agents with varying reliance on retrieved cases — Angel.R (no case reference), Angel.D (retrieved cases displayed as context), Angel.C (comparative analysis of retrieved cases vs. current query) — independently diagnose. If they disagree, a Positive Debate Agent and Negative Debate Agent argue iteratively, with a Judge Agent determining when to conclude and outputting the final diagnosis.

MoodSyn is synthesized via TabSyn (VAE + latent diffusion), with conditional denoising to preserve inter-feature clinical correlations and dynamic loss weighting to maintain scale-score consistency.

Key Contributions

  • First multi-agent framework explicitly targeting mood disorder (depression/bipolar) diagnosis, using item-level granular scale decomposition rather than aggregate scores.
  • Three complementary diagnostic agents (Angel.R/D/C) with a structured Positive/Negative debate + Judge pipeline.
  • MoodSyn: 1,173 synthetic psychiatric cases with 25 features, open-sourced, evaluated across statistical fidelity, ML efficacy, and privacy preservation.
  • Ablation evidence that scale item selection (+6.8% ACC over unfiltered totals) and narrative-format medical records are critical design choices.

Results

Real-world data (n=561): - Angel.R (GPT-4o backbone): ACC 0.920, MCC 0.829 vs. GPT-4o baseline ACC 0.797, MCC 0.492 — +12.3% accuracy - multi-Angels (GPT-4o): ACC 0.925, MCC 0.832 — best overall - multi-Angels (DeepSeek-V3): ACC 0.923, MCC 0.832 - All MoodAngels variants exceed all five baselines (LLaMA3-8B, Mistral-7B, GPT-4o, DeepSeek-V3, MedGemma-27B)

Synthetic data (MoodSyn, n=140): - multi-Angels (DeepSeek-V3): ACC 0.821, Recall 0.824 vs. DeepSeek-V3 baseline ACC 0.821, Recall 0.761

Ablation: - Unfiltered scale totals vs. selected 16 items: ACC drops from 0.920 → 0.852 (−6.8%), MCC from 0.829 → 0.708 (−0.121) - Structured vs. narrative medical record format: ACC 0.914 vs. 0.920 (narrative preferred)

Limitations

  • Framework only accesses medical records and scale scores; does not replicate the ~1-hour clinician interview with real-time behavioral observation.
  • Performance on borderline cases (mild symptom presentations just below diagnostic thresholds) remains unreliable and requires human follow-up.
  • Cases where medical record suggests severe disorder but scale scores show minimal symptoms (e.g., remission or medication effects) cause systematic misdiagnosis.
  • MoodSyn evaluation is internal; generalizability of the synthetic data to other clinical populations/hospitals is untested.
  • Single-hospital dataset limits demographic diversity and external validity.

Relevance to Agentic AI / LLM Agents

MoodAngels demonstrates a domain-specialized multi-agent architecture where tool use (dense retrieval, DSM-5 knowledge base lookup), role differentiation (three agents with calibrated reliance on historical cases), and structured adversarial debate replace monolithic LLM prompting — a design pattern directly applicable to any high-stakes diagnostic or decision-support agent. The debate-and-judge mechanism is a concrete instantiation of multi-agent verification loops, showing measurable gains over single-agent baselines in a setting where factual errors have real consequences. The granular item-selection preprocessing also illustrates how careful input engineering for agents (not just prompt engineering) can dominate raw model capability. For researchers tracking agentic AI, this paper is a worked example of RAG + tool-augmented + multi-agent debate applied to a structured expert domain.