Skip to content

CURV: Coherent Uncertainty-Aware Reasoning in Vision-Language Models for X-Ray Report Generation

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CURV is a three-stage training framework for chest X-ray report generation that jointly models diagnostic uncertainty (both structural and semantic) and explicit clinical reasoning within a VLM. It combines uncertainty-aware supervised fine-tuning, reasoning initialization via a novel dataset (TRACE-CXR), and GRPO-based reinforcement learning with a multi-component reward. The result is a 3B-parameter model that outperforms larger specialist and generalist baselines on standard NLP and clinical accuracy metrics.

Problem

Existing VLMs for radiology report generation focus on factual accuracy but neglect two clinically critical properties: (1) appropriate expression of diagnostic uncertainty (e.g., hedging language like "likely" or "possible" for ambiguous findings) and (2) transparent reasoning chains that logically connect visual findings to clinical impressions. Without these, AI-generated reports lack the nuance and trustworthiness required for clinical adoption.

Method

CURV operates through a three-stage pipeline on Qwen2.5-VL-3B:

  1. Uncertainty-aware SFT (Stage 1): Fine-tunes on 112,111 uncertainty-annotated MIMIC-CXR samples. Ground-truth structural uncertainty specifications encode bounding boxes, anatomical labels, and textual confidence descriptions (e.g., "could be compatible with pneumonia"). Loss is standard MLE over the serialized annotation sequence, teaching the model to detect, localize, and articulate structural uncertainty.

  2. Reasoning initialization (Stage 2): Fine-tunes on TRACE-CXR, a purpose-built dataset of 2,000 CXR reports each augmented with an LLM-generated <thinking> section that bridges findings → impression. This instills the tripartite report structure ⟨Findings, Thinking, Impression⟩.

  3. RL refinement (Stage 3): Uses GRPO on MIMIC-CXR with only ground-truth findings and impression provided (no reasoning supervision), forcing the model to discover coherent reasoning autonomously. The total reward is:

R_total = w_fmt·R_fmt + w_acc·R_acc + w_coh·R_coh

  • R_fmt: Binary reward for strict tripartite structure adherence.
  • R_acc: Weighted combination of RadGraph entity F1 (0.7) and uncertainty alignment via (entity, term, score) triple matching (0.3), averaged across findings and impression.
  • R_coh: Jaccard similarity between RadGraph entities in the <thinking> section and the union of entities in findings + impression, enforcing logical cross-section coherence.

Key Contributions

  • Novel framework combining structural and semantic uncertainty modeling with explicit intermediate reasoning in a unified VLM pipeline for CXR report generation.
  • TRACE-CXR dataset: 2,000 CXR reports with LLM-generated explicit reasoning pathways, validated by a board-certified radiologist; to be publicly released.
  • Multi-component RL reward (R_fmt, R_acc, R_coh) that jointly optimizes format, medical accuracy with uncertainty alignment, and reasoning coherence via GRPO.
  • Demonstration that a 3B-parameter specialized model outperforms 7B generalist/specialist baselines and Gemini 2.5 Pro on clinical metrics.

Results

MIMIC-CXR (in-distribution): - CURV vs. MAIRA-2 (best prior specialist): BLEU-3 9.85 vs. 9.01; METEOR 30.43 vs. 26.78; ROUGE-L 31.19 vs. 28.65; RadGraph F1 19.54 vs. 17.05; CheXbert Micro F1 57.12 vs. 46.53; CheXbert Acc. 76.93 vs. 67.39. - CURV vs. Gemini 2.5 Pro: RadGraph F1 19.54 vs. 7.71; CheXbert Micro F1 57.12 vs. 48.45.

IU X-ray (out-of-distribution, zero-shot generalization): - CURV outperforms all baselines: BLEU-1 29.23 vs. 26.37 (MAIRA-2); METEOR 38.30 vs. 31.02 (HuatuoGPT); ROUGE-L 39.08 vs. 31.18 (MAIRA-2); RadGraph F1 25.65 vs. 24.01 (MAIRA-2); CheXbert Acc. 91.56 vs. 89.89 (HuatuoGPT).

LLM-based qualitative evaluation (RL training progression): - Thinking section Logical Coherence: 2.23 → 4.55; Consistency: 2.88 → 4.76 (by step 3,200). - Semantic uncertainty score: 1.60 → 2.60; Structural uncertainty: 0.60 → 0.76.

Limitations

  • TRACE-CXR reasoning annotations are LLM-generated, not written by radiologists, introducing potential noise in the reasoning supervision signal.
  • Performance depends on the quality of the curated uncertainty-annotated dataset; errors in annotation propagate through Stage 1 SFT.
  • No large-scale prospective clinical validation by expert radiologists has been performed; clinical deployment safety is unconfirmed.
  • Generalization to other imaging modalities (CT, MRI, ultrasound) has not been investigated.
  • Structural uncertainty scores showed non-monotonic learning dynamics during RL (early dip), suggesting optimization instability for fine-grained finding-level uncertainty.

Relevance to Vision-Language Models

CURV advances the frontier of domain-specialized VLMs by demonstrating that integrating uncertainty-aware supervision and structured chain-of-thought reasoning via RL can substantially improve both clinical accuracy and interpretability without scaling model size. The use of GRPO with a domain-specific, multi-component reward is a direct extension of the DeepSeek-R1/Vision-R1 reasoning paradigm to a high-stakes vertical, showing that SFT-then-RL with coherence rewards can elicit emergent reasoning structure in compact VLMs. For researchers tracking VLMs, CURV exemplifies how task-specific reward engineering—particularly reward terms measuring cross-section logical consistency—can substitute for large-scale human-annotated reasoning supervision and generalizes beyond the training distribution.