Skip to content

DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration

🕒 Published (v1): 2025-05-24 10:26 UTC · Source: Arxiv · Venue: EMNLP 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DDO is a multi-agent LLM framework for automated medical consultation that explicitly decouples symptom inquiry (a sequential decision task) from disease diagnosis (a classification task) and optimizes each with a distinct objective. A lightweight RL policy agent narrows the action space for an LLM-based inquiry agent, while a LoRA adapter trained via in-batch contrastive learning sharpens disease-level diagnostic confidence from LLM logits. On three real-world MC datasets, DDO matches or exceeds generation-based SOTA while requiring far less training overhead.

Problem

Existing LLM-based medical consultation methods treat symptom inquiry and disease diagnosis as a single unified optimization target. Because the two sub-tasks have fundamentally different structures—inquiry is a large-action-space sequential MDP; diagnosis is low-cardinality classification—unified training (e.g., Chain-of-Diagnosis instruction tuning) yields conflicting gradients and poor performance on both. Separately, direct LLM prompting produces arbitrary questioning and unreliable confidence scores, especially for symptom-overlapping diseases.

Method

DDO orchestrates four agents over a shared memory (a static disease–symptom knowledge base plus a dynamic symptom/confidence store updated each turn):

  1. Diagnosis Agent: estimates per-disease confidence using Binary Token Probability (BTP)—the LLM is prompted to output a single True/False token per candidate disease; logits at that position are softmax-normalized: \(\text{conf}_i = \frac{\exp(\text{logit}_T/\tau)}{\exp(\text{logit}_T/\tau)+\exp(\text{logit}_F/\tau)}\). A LoRA adapter is trained via in-batch contrastive learning (KL divergence against a smoothed one-hot target) using partial consultation trajectories as weak supervision, improving discrimination among symptom-overlapping diseases.

  2. Policy Agent: a lightweight MLP actor-critic trained with PPO over a masked action space \(A_{\text{masked}} = A \odot M\), where \(M\) disables already-queried or irrelevant symptoms. Reward combines symptom frequency in the ground-truth disease (\(\text{freq}(a_t)\)), presence hit/miss (\(r_{\text{hit}}\)), confidence rank change (\(r_{\text{rank}}\)), and a terminal diagnosis reward. The agent samples \(N\) candidate actions rather than committing to one, offloading the final choice to the LLM.

  3. Inquiry Agent: an LLM that selects the optimal symptom \(s_o\) from the \(N\) candidates via chain-of-thought reasoning (confirm likely diagnosis vs. maximize diagnostic coverage), regenerating the candidate set if none meet criteria.

  4. Patient Agent: simulates patient responses using the MCR profile plus clinical knowledge—inferring presence/absence for undocumented symptoms based on disease typicality.

Key Contributions

  • Formal decoupling of symptom inquiry and disease diagnosis with distinct optimization objectives inside a single framework.
  • BTP-based confidence estimation that extracts diagnostically useful signal from raw LLM logits without numeric generation.
  • LoRA adapter calibrated via in-batch contrastive learning, requiring only ground-truth disease labels (no expert-annotated confidence scores).
  • Masked sampling strategy: RL policy generates a reliable candidate set; LLM selects among candidates rather than over the full action space.
  • Demonstration that RL-LLM collaboration outperforms either agent acting alone in symptom inquiry.

Results

  • DXY (5 diseases, 527 MCRs): DDO 94.2% Acc (Qwen2.5-14B), vs. HAIformer 88.5%, CoD best 60.6%, UoT best 67.3%.
  • GMD (12 diseases, 2,374 MCRs): DDO 80.3% Acc, vs. HAIformer 90.4% (traditional SOTA), CoD best 71.1%.
  • CMD (27 diseases, 5,200 MCRs): DDO 68.6% Acc, vs. HAIformer 64.1%; CoD collapses to 23.0–34.6%.
  • DDO improves over self-reported-only diagnosis (Acc\(_{\text{init}}\)) by an average of +24.6% (DXY), +11.3% (GMD), +3.2% (CMD).
  • Ablation: removing the LoRA adapter drops CMD accuracy by ~9.4 pp; replacing RL-LLM collaboration with LLM-only inquiry causes the largest accuracy drop across datasets; disabling action masking or retry also degrades performance.
  • BTP-adapter outperforms Numerical, Numerical-SC, FirstLogit, and AvgLogit confidence baselines on both GMD and CMD.

Limitations

  • Evaluated only on small, structured MC datasets (DXY/GMD/CMD) with closed disease sets; generalization to open-ended clinical dialogue is unverified.
  • The RL policy agent is trained per-dataset on disease–symptom frequency statistics; porting to a new medical domain requires retraining.
  • Maximum consultation length is capped at \(L=10\) turns; real consultations may require longer or more variable horizons.
  • Patient Agent's inference of undocumented symptoms relies on disease-label leakage at training time; the boundary between training and test knowledge could be blurred in practice.
  • Experiments use relatively small LLM backbones (Qwen2.5-7B/14B); scalability to frontier-size models is not explored.

Relevance to Harnesses / Meta-Harnesses

DDO is a concrete example of a typed multi-agent harness: it defines a fixed workflow DAG in which specialized agents (RL policy, LLM inquiry selector, LLM diagnoser, LLM patient simulator) cooperate via a shared memory bus, with explicit control flow (masked sampling → LLM selection → termination check → diagnosis) rather than free-form agent chaining. The design principle—decouple sub-tasks by objective type and assign each to the agent architecture best suited for it (RL for large discrete search; LLM for language-grounded reasoning; contrastive adapter for metric learning)—directly informs how a meta-harness should route sub-problems to specialized agents. The shared memory pattern (static knowledge + dynamic observation accumulation updated across turns) is a reusable harness primitive for any multi-turn information-gathering pipeline. The BTP logit-extraction technique offers harness designers a lightweight confidence signal from frozen LLMs without external scoring calls.