Skip to content

CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics

🕒 Published (v1): 2026-05-10 14:51 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CLR-voyance reformulates inpatient clinical decision support as a POMDP and trains small LLMs (Qwen3-8B, MedGemma-4B) via GRPO with per-case outcome-grounded rubrics derived from actual patient trajectories in MIMIC-IV. The resulting 8B model scores 84.91% on the authors' held-out benchmark, surpassing GPT-5 (77.83%) and MedGemma-27B (66.66%), and has been deployed at a partner hospital processing 1.03T+ tokens over six months.

Problem

Existing clinical LLM evaluations and RL reward signals collapse the inherently sequential, partially observable nature of inpatient reasoning into three broken paradigms: (1) exam-style MCQ benchmarks where frontier models are saturated and scores are weakly predictive of real-world performance; (2) longitudinal-EHR benchmarks that expose the full admission or embed ground truth in the input document (answer leakage); and (3) static or outcome-agnostic rubric/rule-based RL rewards that do not condition on what a real care team actually did next for that specific patient. No prior system jointly provides outcome-grounded training, clinician validation at every pipeline stage, and verified large-scale hospital deployment.

Method

CLR-POMDP formulation. Each MIMIC-IV admission is modeled as a POMDP M = (S, A, O, T, Z, r). The chronologically ordered event stream (ED, hosp, ICU, notes; 65,536-token context, verbatim JSON) is split at a randomly drawn decision point k ~ N(n/2, n/6), yielding a policy-visible past and oracle-only future. ICD codes are stripped from the past; admissions with re-admission or death within two years are excluded.

Grounded Judge. An oracle LLM with access to the full trajectory generates, for each of four action-space categories (Diagnosis, Treatment, Procedural, Uncertainty): a query answerable from the past alone, a reference response evidenced by future events, and a per-case adaptive rubric. Each rubric criterion is a pass/fail statement on one of five axes (Accuracy, Completeness, ContextAwareness, CommunicationQuality, InstructionFollowing) weighted in [−10, +10]. A separate grading call scores candidate responses; the scalar reward r_rub normalizes positive-weight hits by the sum of positive weights, allowing negative criteria to penalize without inflating the ceiling.

Post-training. Qwen3-8B and MedGemma-4B are post-trained with GRPO (G ∈ {4,8} rollouts per prompt, KL-penalized policy gradient). SFT on oracle references is compared as an ablation. After GRPO, the task vector is merged back into the base model using DELLA-Linear or Breadcrumbs (optionally + Activation-Informed Merging) to recover generalist capability via weight-space interpolation.

Key Contributions

  • CLR-POMDP: first POMDP-style inpatient reasoning benchmark built from raw MIMIC-IV at production context lengths, usable for both training and evaluation without answer leakage.
  • Per-case adaptive rubrics: first rubric-as-reward signal conditioned on a patient's actual downstream care trajectory, enabling dense reward for open-ended management decisions.
  • CLR-voyance-8B: GRPO + DELLA-Linear merging yields state-of-the-art inpatient reasoning at 8B scale, outperforming GPT-5 by 7.08 pp.
  • Clinician validation study: four board-certified physicians validate rubric quality, query validity, judge alignment, and model preference across spine/orthopaedic and general-medicine/obesity cohorts.
  • Hospital deployment: 6+ months in production at a partner public hospital, 1.03T+ tokens, thousands of daily notes, covering Assessment, Plan, Impressions, and Review of Systems elements.

Results

  • CLR-voyance-8B: 84.91% aggregate on CLR-POMDP vs. GPT-5 77.83%, GPT-4.1 75.74%, Qwen3-8B base 75.28%, MedGemma-27B 66.66%, HuatuoGPT-o1-7B 51.24%.
  • Per-axis improvement over GPT-5: Completeness +21.36 pp (80.93 vs. 59.57), Accuracy +11.08 pp (80.73 vs. 69.65), ContextAwareness +4.05 pp; GPT-5 retains +3.38 pp on CommunicationQuality and +0.98 pp on InstructionFollowing.
  • Action-space gains over GPT-5: Procedural +8.0 pp, Uncertainty +7.6 pp, Treatment +6.5 pp, Diagnosis +2.7 pp.
  • GRPO alone reaches 77.95%; DELLA-Linear merging adds +6.96 pp to reach 84.91%; four merge variants span a ±0.7 pp band (84.25–84.91%).
  • SFT on oracle references regresses Qwen3-8B from 75.32% to 52.36%; GRPO reaches 77.99%.
  • MedGemma-4B: base 27.01% → GRPO 63.40% → merge ~63.35% (merge re-balances axes without aggregate gain for 4B).
  • External benchmarks (n=200): CLR-voyance-8B improves over its base on MedMCQA, DDXPlus (+1–9 pp), MedCalc-Bench, MIMIC-Instr, and HealthBench (+0.051 lift on 8B, +0.072 on 4B); only model in the 8B class to exceed GPT-5 on DDXPlus (48.5%).
  • DeepSeek-R1-Distill-Qwen-7B scores 11.99%—order-of-magnitude below all other models—confirming that general-purpose reasoning post-training does not transfer to EHR-based CDS.
  • Clinician preference (blinded): CLR-voyance-8B preferred over GPT-5 at 85.6% [78–91] win rate (spine) and 48.0% [41–55] (obesity); preferred over HuatuoGPT-o1-7B at 94.2% and 82.2% respectively.
  • Grounded Judge (Qwen3-32B) achieves κ=0.42 substantial agreement with pooled clinician verdicts on rubric grading.

Limitations

  • CLR-POMDP is derived from MIMIC-IV (single academic US health system); generalization to other EHR systems and non-US care contexts is unvalidated.
  • The 8B model size is constrained by on-premise GPU infrastructure at the deployment site, not by model capability; larger-scale training is left to future work.
  • The Grounded Judge itself is an LLM; rubric generation and grading introduce oracle noise (Grounded Judge accuracy ~76.7%, κ=0.42 against clinicians).
  • Clinician validation covers only two specialty cohorts (spine/orthopaedic, general-medicine/obesity) with four physicians; broader specialty coverage is needed.
  • Clinical failure modes remain: insufficient drug-drug interaction checking (especially opioids), underweighting of advanced age, occasional patient-info contamination in obesity cases, verbose internal-monologue leakage, and Chinese-character intrusions from Qwen3 base.
  • GPT-5 shows 100% self-preference as judge on GPT-5 vs. CLR-voyance pairs, illustrating LLM-as-judge self-preference bias that complicates head-to-head evaluation.
  • The obesity cohort preference results are mixed (48% win rate vs. GPT-5), suggesting user-persona-dependent requirements (answer-first presentation, resource-constrained workups) not yet captured in training.

Relevance to Foundation Models in Medicine

CLR-voyance directly addresses the core unsolved problem in clinical FM deployment: how to provide a training signal for open-ended inpatient reasoning that is both outcome-grounded and clinician-validated, without requiring prospective data collection. The POMDP framing and per-case adaptive rubric approach are broadly applicable to any clinical FM pipeline that must reason under partial observability—a condition that applies to virtually every real inpatient task. The finding that GRPO on outcome-grounded rubrics outperforms both SFT and frontier proprietary models at 8B scale, combined with 6+ months of production evidence, establishes a credible path toward on-premise deployable clinical reasoning FMs that do not rely on API access to closed models. The clinician alignment study's nuanced findings on judge self-preference, cohort-specific rubric editing, and failure mode taxonomy are directly actionable for future clinical FM evaluation and RLHF pipeline design.