MedLA: A Logic-Driven Multi-Agent Framework for Complex Medical Reasoning with Large Language Models¶

🕒 Published (v1): 2025-09-28 08:06 UTC · Source: Arxiv · Venue: AAAI · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

MedLA is a multi-agent LLM framework for complex medical reasoning in which each agent structures its inference as a syllogism-based logical tree (\(T=(V,E)\), nodes = major-premise/minor-premise/conclusion triads), then engages in multi-round, graph-guided cross-agent revision to resolve premise-level conflicts. It achieves state-of-the-art results on differential diagnosis and general medical QA benchmarks, outperforming prior multi-agent, RAG, and CoT baselines without requiring external retrieval or fine-tuning.

Problem¶

Existing multi-agent medical reasoning systems assign fixed roles and aggregate conclusions via shallow positional discussion; they cannot localize fine-grained logical inconsistencies (conflicting premises, invalid causal links), which limits their ability to correct errors and reduces diagnostic reliability in clinical settings.

Method¶

MedLA decomposes a clinical QA query through a three-phase pipeline:

Phase A — Premise Extraction & Decomposition. A Premise Agent (P-Agent) extracts knowledge-grounded major premises \(P^{\text{maj}}\) (medical rules) and patient-specific minor premises \(P^{\text{min}}\) from the question. A Decompose Agent (D-Agent) recursively splits the question into atomic sub-questions using elimination-based reasoning.
Phase B — Parallel Tree Generation, Calibration & Discussion. \(N_M\) Medical Agents (M-Agents) run in parallel, each independently building a syllogistic logical tree \(T_{M^{(j)}} = (V, E)\) where each node \(v_i\) is a (major premise, minor premise, conclusion) triad. A Credibility Agent (C-Agent) scores each node as high/medium/low confidence. Low-confidence nodes are flagged for discussion; agents exchange their full trees and iteratively revise flagged nodes via a structured revision prompt until convergence.
Phase C — Logical Decision. All local trees are merged; the final answer is generated by traversing the merged tree and aggregating syllogism conclusions.

The framework uses prompt engineering to elicit tree-structured outputs from LLMs (which cannot produce native tree structures), and proves convergence and stability properties in the appendix.

Key Contributions¶

First multi-agent medical reasoning framework representing each agent's inference as an explicit syllogism-based logical tree, enabling premise-level traceability and conflict localization.
Multi-round, graph-guided discussion mechanism where agents compare and revise full logical trees (not just final answers), enabling systematic cross-agent error correction.
Comprehensive evaluation across three benchmark families (MedDDx differential diagnosis, multi-choice medical QA, MedXpertQA expert-level reasoning) on both open-source and commercial LLMs.
Ablation demonstrating additive, independent contributions of the logic tree, credibility calibration, and revision loop components.

Results¶

MedDDx (LLaMA-3.1-8B): MedLA achieves 44.3% average accuracy vs. best baseline MDAgents at 37.7% (+7.4 pp); tiers: Basic 48.2%, Intermediate 43.0%, Expert 41.7%.
Difficulty scaling: Relative improvement over backbone grows monotonically — +4.6 pp (Basic), +6.4 pp (Intermediate), +11.1 pp (Expert).
Multi-choice medical QA (MMLU-Med / MedQA-US / BioASQ-Y/N): MedLA 69.9% average vs. best baseline MDAgents at 60.8% (+5.7 pp above next-best comparable).
Commercial LLMs (MedXpertQA, 60-question subset): MedLA+DeepSeek-R1: 36.0% vs. baseline 21.3%; MedLA+DeepSeek-V3: 25.6% vs. baseline 15.0%.
Backbone scaling: On MedDDx-Expert, MedLA improves LLaMA-3.1-8B from 30.6% to 41.7% (+11.1 pp) and LLaMA-3.1-70B from 41.8% to 51.9% (+10.1 pp).
Ablation (MedQA-US / MedDDx-Basic / Expert): Full MedLA: 62.6 / 48.2 / 41.7; −Revision loop: 58.4 / 44.2 / 38.6; −Credibility: 57.3 / 41.8 / 37.2; −LogicTree (CoT only): 56.1 / 38.7 / 34.9; Majority Vote: 54.8 / 37.5 / 30.2.
Latency: 3,657 seconds wall-clock on BioASQ-Y/N (17 sub-agents), ~2× majority voting, far below KGAREVION (10k+ seconds including fine-tuning); no external retrieval overhead.

Limitations¶

Wall-clock latency is ~2× single-pass multi-agent baselines due to 17 sub-agents and iterative revision rounds; infeasible for real-time clinical deployment at current scale.
LLMs cannot produce native tree-structured outputs; the tree is approximated via prompt engineering, introducing potential structural noise.
Evaluations use multiple-choice question formats; applicability to open-ended generation, clinical note production, or interactive diagnosis is not demonstrated.
Commercial LLM experiments are on a small 60-question subset of MedXpertQA, limiting statistical confidence.
No evaluation against retrieval-augmented methods that also use structured reasoning, leaving open whether combining MedLA with external knowledge would yield further gains.

Relevance to Harnesses / Meta-Harnesses¶

MedLA exemplifies a structured multi-agent harness pattern: it orchestrates four distinct specialized agents (P-Agent, D-Agent, M-Agents × N, C-Agent) in a fixed three-phase pipeline with explicit inter-agent communication topology (tree exchange) and a convergence-driven termination condition — the core design elements of a meta-harness. The credibility-gated revision loop is particularly relevant as a quality-control layer analogous to harness-level verification: it dynamically flags low-confidence intermediate outputs and routes them back for adversarial re-examination rather than passing them downstream unchecked. The ablation showing that the revision loop and credibility calibration each contribute independently (+4.2 pp and +1.1 pp on MedQA-US) provides concrete evidence that harness-level orchestration logic — not just the underlying LLM — is a significant source of performance gain, directly motivating structured harness design over flat agent aggregation.