Skip to content

Transition-Aware best-of-N sampling for Longitudinal Chest X-ray Reports

🕒 Published (v1): 2026-06-23 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Chest X-ray report generators overwhelmingly treat each image in isolation, ignoring the patient's prior visit. This paper proposes a training-free best-of-N selection scheme that scores candidate reports by how well their implied change from the prior report matches the distribution of changes seen in training data. Across three VLMs and three prompting regimes, the method consistently outperforms random candidate selection, with the largest gains on the change-oriented Impression section.

Problem

Existing best-of-N pipelines for radiology report generation score candidates against a reference distribution of single-image reports, ignoring longitudinal context. A candidate that looks individually plausible may still describe a clinically implausible change from the prior exam (e.g., hallucinating acute worsening or silently dropping a chronic finding). No prior work has exploited the prior→current transition signal at the selection stage of a best-of-N pipeline.

Method

Each report is split into sentences and embedded with a frozen sentence transformer (all-mpnet-base-v2, \(d=768\)) into an unordered set \(E(r) \subset \mathbb{R}^d\). A directional set-to-set distance \(T: 2^{\mathbb{R}^d} \times 2^{\mathbb{R}^d} \to \mathbb{R}^{d'}\) reduces each (prior, current) embedding-set pair to a fixed-dimensional transition vector. Four designs are proposed:

  • Mean-Shift: \(T_{\text{ms}}(A,B) = \mu(B) - \mu(A)\)
  • Novelty: \(T_{\text{nov}}(A,B) = \frac{1}{|B|}\sum_{b\in B}(b - \text{NN}_A(b))\) (forward Chamfer residual)
  • Dir-Hausdorff: displacement at the single worst-covered point in \(B\) relative to \(A\)
  • Cost-OT: \(T_{\text{cot}}(A,B) = \sum_{i,j} \gamma^*_{ij} M_{ij}(b_j - a_i)\) where \(\gamma^*\) is the optimal transport plan

At training time, a bank \(\mathcal{B}_T = \{g_t\}\) of ground-truth transition vectors is cached once. At test time, \(K=5\) candidates are generated; each candidate's transition vector \(\hat{g}^{(j)}\) (computed against the patient's actual prior) is scored via cosine distance to the bank using either \(D_{\min}\) (nearest single entry) or \(D_{k\text{NN}}\) (mean of \(k\) nearest). The candidate minimising \(D(\hat{y}) = D_F(\hat{y}) + D_I(\hat{y})\) is selected.

Key Contributions

  • First training-free best-of-N scoring scheme that explicitly conditions on the longitudinal prior→current transition rather than individual report quality.
  • Four directional set-to-set distance instantiations (mean-shift, novelty residual, directed-Hausdorff, cost-weighted OT) with two aggregation modes (min, kNN).
  • A reusable transition-vector bank built once from training data; no model fine-tuning required and compatible with any frozen VLM generator.
  • Empirical evaluation across 3 VLMs Ă— 3 prompts on a 1,727-transition test set drawn from ReXGradient-160K.

Results

Impressions (averaged across all 9 modelĂ—prompt runs vs. random baseline): - Best ROUGE-L: +8.6% (Novelty/kNN) - Best RadGraph F1: +13.6% (Mean-Shift/min) - Best METEOR: +4.4% (Dir-Hausdorff/kNN) - Mean across all 8 transition variants: +6.5% ROUGE-L, +11.3% RadGraph, +3.4% CheXbert-14

Findings (weaker gains, as expected for the less change-oriented section): - Best RadGraph F1: +4.7% (Cost-OT/min) - Best BERTScore: +2.5% (Mean-Shift/min, Cost-OT/min) - Mean across variants: +1.3% ROUGE-L, +3.9% RadGraph - Every transition variant beats random on every headline Findings metric.

Limitations

  • Gains on the Findings section are modest (≤+4.7%), likely because Findings are less change-oriented than Impressions.
  • The transition bank captures only single-step prior→current changes; longer patient histories are not exploited.
  • Candidate pool is fixed at \(K=5\); scaling behavior with larger \(N\) is unexplored.
  • All directional distances use hand-designed forms; learned encoders optimised for clinical metrics are left as future work.
  • Generator diversity is limited to two model families (Gemini variants, Mistral-Small); generalization to other VLMs is unverified.
  • The method requires access to the patient's prior report at inference time, which may not always be available.

Relevance to Vision-Language Models

This work directly benchmarks three VLMs (Gemini-2.5-Flash-Lite, Gemini-3.1-Flash-Lite-preview, Mistral-Small-2603) as chest X-ray report generators and demonstrates that training-free test-time selection in transition space consistently improves their outputs without any weight updates. For researchers tracking VLMs, it establishes that post-hoc geometric scoring in sentence-embedding space is a viable and compute-light complement to the more expensive approaches of fine-tuning or prompting for longitudinal context. The framework is generator-agnostic, making it immediately applicable to any future VLM deployed in clinical radiology pipelines.