Skip to content

A Guideline-Aware AI Agent for Zero-Shot Target Volume Auto-Delineation

🕒 Published (v1): 2026-03-10 10:00 UTC · Source: Arxiv · Venue: MICCAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

OncoAgent is a training-free AI agent that converts free-text clinical radiotherapy guidelines into 3D target volume contours by planning a tool-call sequence (LLM reasoning) and executing it via pre-trained OAR segmentation models and geometric operations. On esophageal cancer, it achieves zero-shot DSC of 0.842 (CTV) and 0.880 (PTV), statistically indistinguishable from a fully supervised nnU-Net baseline. Physicians preferred it over the supervised baseline on guideline compliance and clinical acceptability.

Problem

Deep learning segmentation models (nnU-Net, DDAU-Net) for clinical target volume (CTV) delineation are data-hungry black boxes: they require large expert-annotated datasets and must be fully retrained whenever clinical guidelines are updated, making them brittle to the frequent protocol changes in radiotherapy practice.

Method

OncoAgent operates in two phases. In the planning phase, a system prompt encoding three strategies—clinician role assignment (with mandatory templates \(\text{CTV} = (\text{GTV} \oplus m_\text{ctv}) \setminus \text{OARs}\), \(\text{PTV} = \text{CTV} \oplus m_\text{ptv}\)), guideline parameterization (OAR alias resolution, margin range disambiguation), and robust tool-call sequencing (algorithmic prompting for ordered segment→union→dilate→subtract steps)—instructs an LLM (GPT-5.2) to parse a raw clinical guideline and emit a JSON tool-call sequence. The plan is validated against a JSON schema; if invalid, the LLM self-refines iteratively. In the execution phase, a deterministic engine dispatches calls sequentially: a pre-trained OAR segmentation DL model delineates anatomical structures on patient CT, and VTK-based geometric tools apply direction-specific dilation and Boolean subtraction. Post-processing (contour smoothing, hole-filling) finalizes the volume. Average runtime is 1.74 min/patient with a mean of 1.13 LLM calls per case.

Key Contributions

  • Two-phase agentic architecture (plan → execute) that converts free-text guidelines to 3D contours with zero CTV-annotated training data.
  • Demonstrated zero-shot performance (DSC 0.842/0.880 for CTV/PTV) statistically equivalent to fully supervised nnU-Net (GTV Prior) (DSC 0.862/0.893) on 8 esophageal test cases.
  • Blinded clinical preference for OncoAgent over supervised baseline: Guideline Compliance 4.06 vs. 3.56, Modification Effort 4.12 vs. 3.44 (5-point Likert).
  • Cross-guideline zero-shot generalization to CROSS Trial and JASTRO 2024 esophageal protocols, and to prostate (RTOG 0126), with Tool Call F1 of 0.73–1.00 (esophagus) and 0.64 (prostate).
  • Interpretable intermediate plan (human-readable tool-call sequence) reviewable by clinicians before execution.

Results

  • CTV DSC: OncoAgent 0.842 ± 0.026 vs. nnU-Net (GTV Prior) 0.862 ± 0.026 — no statistically significant difference (\(p \geq 0.05\)); both far above CT-only baselines (DDAU-Net 0.463, nnU-Net 0.627).
  • PTV DSC: OncoAgent 0.880 ± 0.015 vs. nnU-Net (GTV Prior) 0.893 ± 0.021 — no significant difference.
  • MSD: OncoAgent CTV 1.06 mm vs. nnU-Net (GTV Prior) 0.95 mm; PTV 1.12 mm vs. 1.02 mm — sub-voxel in both cases.
  • Sensitivity advantage: OncoAgent higher Sensitivity than supervised model (CTV 0.884 vs. 0.845; PTV 0.922 vs. 0.873), though lower Precision (CTV 0.807 vs. 0.882).
  • Physician ratings (5-point Likert, \(n=8\) cases, 2 senior oncologists): OncoAgent beats nnU-Net (GTV Prior) on all three criteria; Ratings ≥4 are 81.2% vs. 56.2% (compliance), 68.8% vs. 37.5% (modification effort), 75.0% vs. 50.0% (acceptability).
  • Cross-guideline Tool Call F1: IJROBP 1.00, JASTRO 2024 0.78, CROSS Trial 0.73, Prostate RTOG 0126 0.64.

Limitations

  • OAR segmentation quality is a hard upstream dependency; errors propagate directly to the CTV.
  • LLM susceptibility to semantic hallucination and misinterpretation of guideline clauses; implicit clinical knowledge not encoded in text is inaccessible.
  • Strict Boolean OAR subtraction can over-erode the CTV near structures like the heart; no adaptive margin strategy for esophageal motion (respiration, swallowing, cardiac pulsation).
  • Small single-institution cohort (40 patients, 8 test cases); no multi-center validation.
  • Reduced Tool Call F1 (0.64) on prostate due to unfamiliar anatomical terminology not covered by the alias-resolution prompting.

Relevance to Harnesses / Meta-Harnesses

OncoAgent is a concrete domain-specific harness: it wraps heterogeneous pre-trained tools (a DL segmentation model, geometric operation library) behind an LLM planning layer that converts unstructured natural-language specifications (clinical guidelines) into validated, executable tool-call sequences. The plan-validate-self-refine loop and JSON schema enforcement are exactly the schema-grounded tool orchestration patterns central to agentic harness design. The framework's instant adaptability to new guidelines by swapping a text document—without touching tool implementations—demonstrates the meta-harness principle of separating the reasoning/orchestration layer from the execution layer. Cross-domain generalization (esophagus→prostate) from a single harness architecture illustrates how harness-level abstraction can reduce the per-task engineering burden.