Skip to content

RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows

🕒 Published (v1): 2025-09-24 19:08 UTC · Source: Arxiv · Venue: MIDL 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RadAgents is a seven-agent framework that encodes radiologist-style ABCDE review workflows into a modular, auditable pipeline for chest X-ray interpretation. An Orchestrator routes queries to specialized subagents; a Synthesizer aggregates outputs and resolves cross-tool conflicts via Visual RAG. Using an 8B open model, it matches or surpasses GPT-4o and specialist CXR models across three benchmarks.

Problem

Existing CXR AI systems follow an "encode-once, text-only" paradigm: visual reasoning is decoupled from text generation, tool calls are ad hoc, cross-tool inconsistencies go unresolved, and reasoning traces are opaque and not aligned with clinical guidelines. Iterative re-inspection, quantitative measurements, and multi-view/longitudinal comparisons—core to radiologist practice—are poorly supported.

Method

RadAgents decomposes interpretation into seven specialized agents instantiated with Qwen3-VL-Instruct (4B/8B/30B):

  • Five ABCDE subagents (Airway, Breathing, Circulation, Diaphragm, Everything-else) each carry a domain-specific skill set of predefined tool chains with clinical decision thresholds (e.g., CTR calculation, carinal angle measurement). Subagents execute in parallel via ReAct-style reasoning with local re-planning on failure.
  • Orchestrator analyzes query intent, extracts clinical entities, drafts a high-level plan, and dispatches tasks to the relevant subagents only. A three-layer abstraction (workflow → skill → tool) decouples planning from implementation.
  • Synthesizer aggregates subagent outputs, detects inter-tool conflicts, and resolves them via V-RAG: it retrieves \(k\) clinically similar CXR exemplars using Rad-DINO image embeddings and uses them as reference standards to adjudicate disagreements.

A shared short-term memory caches patient context and tool outputs, preventing redundant tool invocations. Workflow-free ReAct fallback handles out-of-template queries.

Key Contributions

  • Formalization of radiologist-like ABCDE multimodal workflows encoded as auditable, skill-layered agent behaviors supporting both workflow-guided and free-form modes.
  • Traceable tool-augmented multi-agent architecture with an Orchestrator–Synthesizer global controller, explicit step-level artifact logs, and V-RAG conflict resolution.
  • Demonstration that an 8B open model in a structured multi-agent harness matches or exceeds GPT-4o and specialist CXR models; ablations isolate the value of workflows, V-RAG, and Synthesizer capacity.

Results

  • ChestAgentBench (2,500 questions, 7 categories): RadAgents 73.6% overall vs. CheXagent 39.5%, GPT-4o 56.4%, Qwen3-VL+ReAct 61.3%, Qwen3-VL+Workflow 63.5%, RadAgents w/o V-RAG 66.9%. Largest gains on Diagnosis (+9.6 pp vs. Qwen3-VL+Workflow) and Characterization (+9.5 pp).
  • CheXbench (VQA + reasoning): RadAgents 74.6% overall vs. CheXagent 64.7%, GPT-4o 63.5%; Rad-Restruct 76.5%, SLAKE 89.4%, OpenI reasoning 69.2%.
  • MIMIC-CXR report generation (GREEN score, multi-view/longitudinal, 2,231 cases): RadAgents 51.4 vs. RadAgents w/o V-RAG 46.1, GPT-4o+Workflow 41.7, GPT-4o+ReAct 42.3, plain GPT-4o 34.2, CheXagent 23.6.
  • V-RAG ablation: contributes ~6–7 absolute points on ChestAgentBench; \(k=3\) balances helpful vs. harmful retrieval rate.
  • Scale ablation: Synthesizer conflict-resolution rate rises from 26.3% (4B) to 44.7% (8B) to 60.5% (30B); Orchestrator dispatch success improves more modestly (87% → 93%).

Limitations

  • Tool capability is a hard ceiling: tools optimized for frontal-view X-rays degrade on lateral views.
  • Orchestrator–tool interaction is unidirectional; the Synthesizer cannot iteratively correct upstream tool outputs (e.g., refine an imperfect segmentation mask).
  • Multi-agent architecture incurs higher computational cost than end-to-end models; running all five subagents at 30B scale is prohibitively expensive.
  • V-RAG beneficial rate plateaus and harmful rate grows with larger \(k\); retrieval quality is capped by embedding model and dataset coverage.
  • No prospective clinical validation; evaluation is purely benchmark-based.

Relevance to Harnesses / Meta-Harnesses

RadAgents is a concrete example of a domain-specific meta-harness: it composes a fixed set of specialized agents, tools, and retrieval modules into a structured pipeline governed by a global controller (Orchestrator + Synthesizer) rather than leaving orchestration to an unconstrained LLM. The three-layer abstraction (workflow → skill → tool) is a reusable harness design pattern that decouples planning from execution, enabling independent evolution of each layer—directly applicable to general meta-harness architectures. The Synthesizer's V-RAG conflict resolution demonstrates how a harness can enforce cross-agent consistency, a problem common to any multi-tool harness. The ablation methodology—upgrading one component at a time while holding others fixed—provides a principled blueprint for evaluating which harness layer bottlenecks overall system performance.