MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis¶

🕒 Published (v1): 2025-06-04 09:18 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

MoodAngels is a retrieval-augmented multi-agent framework for mood disorder diagnosis that decomposes psychiatric scale scores to item-level granularity and synthesizes three specialized diagnostic agents via a structured debate mechanism. It introduces MoodSyn, a 1,173-case synthetic dataset generated via VAE+diffusion to enable privacy-safe research. The baseline single agent (Angel.R) already outperforms GPT-4o by 12.3% accuracy on 561 real clinical cases.

Problem¶

Psychiatric AI diagnosis lacks objective biomarkers, suffers from symptom overlap across disorders, depends on unreliable self-reported scales, and faces a data scarcity bottleneck due to patient privacy constraints. General-purpose LLMs prompted with aggregate scale scores lose intra-scale symptom patterns critical for distinguishing mood disorders from comorbidities.

Method¶

MoodAngels decomposes 13 psychiatric scales into item-level responses, then uses Pearson correlation against mood disorder labels to select the top 5% most discriminative questions, clustering them into five groups (depressive mood, loss of interest, anxiety, insomnia, suicidal ideation). A retrievable knowledge base is built from structured DSM-5 criteria and 2,243 anonymized clinical cases; BGE-M3 dense embeddings retrieve the top-\(k\) most similar criteria and prior cases.

Three agents with varying historical dependence are instantiated: - Angel.R: no case retrieval, DSM-5 criteria only - Angel.D: retrieved cases displayed as context - Angel.C: retrieved cases compared against the query and returned as analytical summaries

Their independent diagnoses feed into multi-Angels: a Judge Agent checks for consensus; on disagreement, a Positive and Negative Debate Agent argue, with the Judge terminating debate when sufficient evidence accumulates.

MoodSyn is synthesized via TabSyn (variational autoencoder + diffusion model) with adaptive tokenization, hierarchical encoding, conditional denoising, and dynamic loss weighting to preserve inter-feature clinical correlations.

Key Contributions¶

First psychiatry-specific multi-agent diagnostic framework targeting mood disorders, with granular item-level scale analysis and structured debate synthesis
Three diagnostic agents with graduated reliance on historical case retrieval, mitigating both under- and over-dependence on precedent
MoodSyn: 1,173 open-source synthetic psychiatric cases with statistical fidelity, ML utility, and stronger privacy guarantees than anonymization
Ablation demonstrating that narrative-format medical records outperform structured formats for atypical presentations, and that item-level scale selection is independently beneficial

Results¶

Real-world data (561 cases): - Angel.R\(_{\text{GPT-4o}}\): ACC 0.920, MCC 0.829, Macro F1 0.913 (+12.3% ACC vs. GPT-4o baseline 0.797) - multi-Angels\(_{\text{GPT-4o}}\): ACC 0.925, MCC 0.832, Macro F1 0.917 (best GPT-4o variant) - multi-Angels\(_{\text{DeepSeek-V3}}\): ACC 0.923, MCC 0.832, Macro F1 0.916 - All agent variants surpass all five baseline LLMs (LLaMA3-8B, Mistral-7B, GPT-4o, DeepSeek-V3, MedGemma-27B)

Synthetic data (140 cases): - multi-Angels\(_{\text{DeepSeek-V3}}\): ACC 0.821, MCC 0.642, Macro F1 0.821 - Angel.C slightly underperforms Angel.D on real data (caution against over-reliance on retrieved cases)

Limitations¶

Evaluated on a single hospital's dataset; external generalizability is unverified
Angel.C's comparative retrieval occasionally degrades accuracy relative to Angel.D, suggesting retrieved-case comparison can mislead rather than clarify
MoodSyn synthetic cases sourced from the same institution as the real data, limiting distributional diversity
Framework is binary (mood disorder vs. not); fine-grained subtype classification (depression vs. bipolar) is not addressed
Debate mechanism termination criteria are not formally specified, leaving convergence behavior implicit

Relevance to Harnesses / Meta-Harnesses¶

MoodAngels is a concrete instance of a task-specific multi-agent harness: it composes three specialized sub-agents with heterogeneous retrieval configurations, a judge agent, and a debate loop—exactly the orchestration pattern that meta-harnesses abstract and generalize. The structured debate-and-judge termination loop is a domain-constrained analogue to the adversarial verification and synthesis phases common in general meta-harnesses. The framework also demonstrates a reusable design principle: graded agent diversity (no-reference → display → comparative) as a mechanism to control the influence of retrieved context, which is transferable to any RAG-augmented harness architecture. For researchers building harnesses, the ablation on medical record format (narrative vs. structured) is a concrete reminder that upstream data representation choices propagate through to agent reasoning quality.