Discrete Diffusion Language Models for Interactive Radiology Report Drafting¶

🕒 Published (v1): 2026-07-01 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper adapts DiffusionGemma-26B, a discrete diffusion language model, for radiology report generation and benchmarks it against its autoregressive sibling Gemma-4-26B under identical LoRA finetuning. The diffusion model matches or exceeds the AR model on medical VQA while decoding 3.5–4.4× faster. Crucially, its bidirectional denoising enables any-order infill — filling arbitrary gaps in a report from fixed text on both sides — a capability structurally unavailable to autoregressive models.

Problem¶

Medical foundation models, including all leading radiology report generation systems (MAIRA, ReXrank), are autoregressive, conditioning each token only on preceding context. This prevents interactive drafting workflows where a radiologist fixes report fragments at arbitrary positions and wants the model to fill gaps conditioned on text on both sides. Prior diffusion models for chest X-ray reports generate only complete reports and do not expose or evaluate this infill capability.

Method¶

Matched backbone comparison. DiffusionGemma-26B (uniform-state discrete diffusion) and Gemma-4-26B (AR), both 25.2B/3.8B-active MoE with a shared SigLIP-lineage vision encoder (~280 image tokens), are finetuned under an identical LoRA recipe (rank-64, $\alpha=128$, targets: $\{q, k, v, o, \text{mlp}\}$, vision tower frozen). Only the generative paradigm and its established optimizer differ (AdamW $\beta=(0.9, 0.95)$ for diffusion vs. $\beta=(0.9, 0.999)$ for AR). The diffusion objective replaces a random fraction of target tokens with uniform vocabulary draws (not [MASK]) and trains the model to recover them.

Any-order infill. At inference, let $F$ be fixed positions with tokens $a$ and $\bar{F}$ the free positions. The model samples: $$x_{\bar{F}} \sim p_\theta(x_{\bar{F}} \mid x_F = a, c)$$ by clamping fixed tokens on the incoming canvas and both outgoing canvases at each denoising step, with no retraining. Because attention within the 256-token canvas is fully bidirectional, free positions condition on fixed tokens to their right as well as left.

Evaluation. Medical VQA accuracy is scored by an LLM judge (Claude Sonnet 4.6, semantic-equivalence) on three datasets (VQA-RAD, SLAKE, VQA-Med-2019), $n=350$ held-out items each. Infill is evaluated on MIMIC-CXR ($n=249$) by masking one sentence and scoring fills by token-F1 and LLM judge under bidirectional vs. left-only context.

Key Contributions¶

First controlled comparison of diffusion vs. AR for medical VQA using matched backbones (same family, size, vision tower, data, LoRA recipe).
Demonstrates that a discrete diffusion LM is a viable medical foundation model: equal or better accuracy than its AR sibling at 3.5–4.4× lower latency.
Formalizes any-order infill as sampling $p_\theta(x_{\bar{F}} \mid x_F = a, c)$ and shows via a simple sampler patch that diffusion exploits bidirectional context significantly while prompted AR does not.
Releases code and finetuned checkpoints.

Results¶

Medical VQA (LLM-judge accuracy, $n=350$/dataset): - Finetuned diffusion equals or exceeds finetuned AR on all three datasets: VQA-RAD 0.649 vs. 0.649, SLAKE 0.863 vs. 0.817 (significant, $p=0.026$), VQA-Med 0.666 vs. 0.631. - Base diffusion exceeds base AR on all three: VQA-RAD 0.614 vs. 0.523 ($p<0.001$), SLAKE 0.700 vs. 0.674, VQA-Med 0.629 vs. 0.614. - Finetuned diffusion exceeds GPT-4.1-mini on all datasets; Gemini-3.5-Flash leads on VQA-RAD (0.777) and VQA-Med (0.683); finetuned diffusion leads on SLAKE (0.863).

Inference speed (one H100, ~256-token generation): - DiffusionGemma at 16 steps: 1.46 s, 175.3 tok/s — 4.4× faster than Gemma-4 AR (6.43 s, 24.6 tok/s). - DiffusionGemma at 48 steps: 1.84 s — 3.5× faster.

Any-order infill on MIMIC-CXR ($n=249$): - Diffusion bidirectional vs. left-only: $\Delta$ token-F1 = $+0.109$ ($p<10^{-10}$), $\Delta$ judge = $+0.129$ ($p=2\times10^{-5}$). - AR bidirectional (right context in prompt) vs. left-only: $\Delta$ token-F1 = $+0.031$ (n.s., $p=0.08$), $\Delta$ judge = $-0.016$ (n.s.). - Model × context interaction: $+0.078$ token-F1 ($p=2\times10^{-4}$), $+0.145$ judge ($p=3\times10^{-4}$); diffusion benefits ~3.5× more from bidirectional context.

Limitations¶

Infill evaluation uses only MIMIC-CXR (chest X-ray); generalization to other modalities or report structures is untested.
The 256-token canvas fits one full report, but longer reports or multi-image contexts are not addressed.
The closed-question accuracy advantage of diffusion over AR may partly reflect answer-format differences rather than purely image-grounding gains.
Frontier VLM comparisons are zero-shot only; finetuned frontier baselines are absent.
The LLM judge (Claude Sonnet 4.6) is also one of the evaluated frontier models, introducing a potential self-evaluation bias (flagged by the authors with a †).

Relevance to Vision-Language Models¶

This work directly challenges the autoregressive assumption underlying virtually all medical VLMs by showing that a diffusion-based VLM (DiffusionGemma-26B) achieves parity or better accuracy on medical visual question answering tasks while offering significantly faster decoding. The any-order infill capability is a structural property of bidirectional decoding unavailable to standard VLMs, opening a new paradigm for interactive, clinician-in-the-loop multimodal generation. For researchers tracking VLMs, this establishes discrete diffusion as a credible alternative backbone for multimodal tasks and raises questions about whether bidirectional generation provides latent advantages for tasks requiring fine-grained image-text grounding beyond what left-to-right factorization allows.