Skip to content

One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction

🕒 Published (v1): 2026-03-31 18:00 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CAMP (Case-Adaptive Multi-agent Panel) addresses case-level heterogeneity in clinical LLM prediction by having an attending-physician agent dynamically assemble a specialist panel per case, collect three-valued votes (KEEP/REFUSE/NEUTRAL), and resolve disagreements through hierarchical evidence-based arbitration rather than majority voting. On MIMIC-IV diagnostic prediction and brief hospital course (BHC) generation, CAMP consistently ranks first or second across four LLM backbones while consuming fewer tokens than most competing multi-agent methods.

Problem

LLM outputs for clinical prediction exhibit non-uniform instability: simple cases yield consistent predictions across prompt variants, while complex or ambiguous cases diverge substantially—a phenomenon the authors term case-level heterogeneity. Existing single-agent methods (CoT, self-consistency) sample from a single role-conditioned distribution, so diversity reflects sampling noise rather than genuine specialist disagreement. Multi-agent frameworks assign static roles across all cases and collapse disagreement via flat majority voting, discarding the diagnostic signal embedded in the disagreement itself. No prior method jointly provides (1) case-adaptive agent composition and (2) structured, reasoning-informed disagreement resolution.

Method

CAMP operates in four stages:

  1. Initial assessment: An attending-physician agent preprocesses the clinical note into structured sections, generates candidate diagnoses \(\{Å·_j^0\}_{j=1}^n\) with ACCEPT/REJECT labels via \(M_\text{att}(x, Y)\), and identifies key clinical dimensions (organ systems, ambiguous findings, diagnostic uncertainty sources). This serves as a low-cost fallback.

  2. Case-adaptive panel assembly: The attending physician assembles \(k=3\) specialist agents \(R = \{r_1,\ldots,r_k\}\) where each \(r_i\) is a role-conditioned prompt pairing a specialty identity with a case-specific focus directive (e.g., "pay attention to nonvisualization of the right vertebral artery"). Panel composition varies per case.

  3. Specialist deliberation: Each specialist \(A_i\) produces per-diagnosis structured outputs \((v̂_{ij}, q_{ij}, c_{ij}, s_{ij})\) — a three-valued vote \(v̂_{ij} \in \{\text{KEEP}, \text{REFUSE}, \text{NEUTRAL}\}\), an evidence quote \(q_{ij}\) from the note, a confidence score \(c_{ij} \in [0,1]\), and a natural-language rationale \(s_{ij}\).

  4. Hybrid resolution: A router dispatches each candidate diagnosis \(d_j\) through one of three paths based on vote counts \((k_j, r_j, n_j)\):

  5. Strong consensus (\(k_j \cdot r_j = 0\) and \(\max(k_j, r_j) > n_j\)): apply directly.
  6. Weak consensus (\(k_j \cdot r_j = 0\) and \(\max(k_j, r_j) \le n_j\)): fall back to attending physician's initial judgment \(Å·_j^0\).
  7. Conflict (\(k_j > 0\) and \(r_j > 0\)): the attending physician performs evidence-based arbitration—examining each specialist's vote, quote, confidence, and rationale—and renders a final ACCEPT/REJECT by weighing argument quality.

For BHC generation, accepted diagnoses \(\hat{Y}\) are passed to the attending physician to generate the narrative grounded in the deliberative output rather than raw notes.

Key Contributions

  • CAMP framework combining case-adaptive specialist assembly with hierarchical arbitration, the first method to jointly address both gaps.
  • Three-valued voting (KEEP/REFUSE/NEUTRAL) enabling principled abstention; high NEUTRAL proportion is itself a signal triggering the fallback path.
  • Hybrid router with three resolution paths (consensus / fallback / arbitration) that reserves expensive LLM arbitration calls for genuinely contested diagnoses.
  • Empirical validation on MIMIC-IV across four LLM backbones on both classification (diagnostic prediction) and generation (BHC) tasks.

Results

  • Diagnostic prediction (Macro F1): CAMP ranks first on 3 of 4 backbones.
  • Llama-3.1-70B: CAMP 87.50 F1 / 67.10 Perfect Rate vs. next-best Majority Voting 85.84 / 66.55.
  • GPT-OSS-20B: CAMP 82.20 / 63.15 vs. CoT 76.54 / 60.50 (+5.66 F1, largest absolute gap).
  • Gemma-3-27B: CAMP 85.10 comparable to Devil's Advocate (85.56) and MedAgents (85.34).
  • GPT-5.4: CAMP 91.31 / 75.65 vs. Single Agent 89.95 / 74.65.
  • BHC generation (rank, lower = better): CAMP achieves best ranks on Llama-3.1-70B across Reasoning (2.49), Readability (2.78), Utility (2.73) vs. Majority Voting 3.43 / 3.53 / 3.55.
  • Specialist-service alignment: Attending physician selects matching specialist without access to service labels — 70% psychiatry, 50% OB/GYN, 35–41% for neurosurgery/neurology/orthopaedics.
  • Panel size sensitivity: F1 rises from 64.84 (\(k=1\)) to 68.97 (\(k=3\)), then plateaus through \(k=7\) with no clear upward trend; precision gain is largest (+8.1%), indicating specialists primarily filter erroneous diagnoses.
  • Majority Voting on GPT-OSS-20B (74.00 F1) falls below Single Agent (75.78), confirming that naive multi-agent aggregation can hurt.

Limitations

  • Full baseline comparisons on GPT-5.4 omitted due to API cost constraints.
  • BHC evaluation relies on an LLM judge (GPT-5-mini rankings), introducing model-as-judge bias.
  • Fixed panel size \(k=3\) across all cases; dynamic \(k\) based on case complexity is not explored.
  • Specialist alignment analysis does not establish a causal link between selection quality and prediction improvement.
  • NEUTRAL-dominated cases fall back entirely to the attending physician's initial (pre-deliberation) judgment, potentially discarding partial specialist signals.
  • Evaluation limited to MIMIC-IV (single institution, 2008–2019); generalizability to other EHR systems or clinical settings is not assessed.

Relevance to Harnesses / Meta-Harnesses

CAMP is a concrete instantiation of the orchestrator-subagent meta-harness pattern: a coordinating agent (attending physician) dynamically selects, prompts, and routes work to specialized sub-agents, then synthesizes their structured outputs via tiered arbitration logic rather than flat aggregation. The hybrid router is a multi-path dispatch harness with explicit branching conditions (\(k_j\), \(r_j\), \(n_j\) thresholds), demonstrating how structured intermediary representations (vote matrices with evidence quotes) enable interpretable, auditable harness behavior. The case-adaptive composition principle—where the orchestrator inspects input properties to choose sub-agent identities at runtime—is directly transferable to non-clinical harness designs where workload heterogeneity warrants dynamic agent selection. The paper also quantifies the cost-quality tradeoff of reserving expensive LLM calls for contested paths, a design consideration central to efficient meta-harness construction.