Skip to content

IMACT-CXR: An Interactive Multi-Agent Conversational Tutoring System for Chest X-Ray Interpretation

🕒 Published (v1): 2025-11-19 19:32 UTC · Source: Arxiv · Venue: ISBI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

IMACT-CXR is an AutoGen-orchestrated multi-agent intelligent tutoring system for chest X-ray interpretation that integrates spatial bounding-box validation, eye-gaze analytics, Bayesian Knowledge Tracing (BKT), PubMed retrieval, and a local vision-language reasoning model (NV-Reason-CXR-3B) into a single conversational tutoring loop. It improves on prior single-agent VLM approaches by adding formal mastery tracking, anatomically aware gaze feedback, and safety mechanisms that prevent premature ground-truth disclosure. Preliminary single-participant evaluation shows higher localization accuracy (mIoU 0.59 vs. 0.51) and faster mastery acquisition (4.2 vs. 6.1 turns) than baselines.

Problem

Existing radiology education systems either rely on static quizzes without adaptive feedback, or leverage vision-language models (e.g., Radiology-GPT, CheXagent) that lack formal multi-agent orchestration, spatial focus validation, eye-gaze integration, and explicit skill mastery tracking. No prior end-to-end system combines real-time spatial annotations with adaptive multi-agent tutoring and explicit focus gating for chest X-ray training.

Method

IMACT-CXR implements a fixed-order sequential AutoGen workflow executed each conversational turn:

  1. Focus Gate: IoU validation (threshold 0.6) of student bounding boxes against ground-truth; turn terminates with directional guidance on failure.
  2. Assessment + Mastery: GPT-4 agent grades free-text interpretation; per-skill BKT updates posterior mastery P(L_t) using (p_init=0.2, p_learn=0.15, p_guess=0.2, p_slip=0.1).
  3. Gaze Coaching: TensorFlow U-Net segments lung lobes from DICOM; coverage ratio, dwell time ratio, and Levenshtein-based sequence score are computed over fixation sequences.
  4. Routing: Based on mastery deltas, explicit student queries, and assessment output, the orchestrator conditionally triggers: SocraticTutorAgent (open-ended coaching with sanitized findings), KnowledgeBaseAgent (PubMed E-utilities retrieval, triggered when mastery < 0.3 after ≥3 attempts), ReasoningAgent (NV-Reason-CXR-3B local inference, triggered when mastery < 0.2 after ≥5 attempts or on explicit request), and CaseSimilarityAgent (REFLACX case retrieval combining label overlap and spatial proximity).
  5. Faculty Response: FacultyStyleResponder fuses all agent outputs into a single natural-language message; safety prompts sanitize ground-truth labels, gate on progressive disclosure, and prohibit exact value echoing.

Key Contributions

  • AutoGen-based sequential multi-agent orchestration unifying spatial, gaze, and textual modalities in a single turn
  • Anatomically aware gaze coaching via U-Net lung-lobe segmentation with coverage, dwell, and sequence metrics
  • Per-skill BKT mastery tracking driving adaptive knowledge reinforcement and reasoning triggers
  • REFLACX-indexed case similarity retrieval for deliberate practice
  • Safety mechanism stack (ground-truth sanitization, progressive disclosure, no-value-echo) preventing answer leakage

Results

  • Localization (mIoU): IMACT-CXR 0.59 vs. Radiology-GPT 0.51 vs. text-based tutor 0.43 (single participant, 20 cases)
  • IoU gate pass rate: 63% vs. 45% vs. 38%
  • Diagnostic reasoning accuracy (end): 0.71 vs. 0.62 (Radiology-GPT) vs. 0.54 (text-based)
  • Turns to mastery: 4.2 (IMACT-CXR) vs. 5.3 (Radiology-GPT) vs. 6.1 (text-based)
  • Mastery progression: mean ΔP = +0.38
  • Ablation contributions (removing component vs. full system):
  • -Gaze: mIoU −6.8%, accuracy −5.6%, turns +0.4
  • -BKT: turns to mastery +21.4%, accuracy −8.5%
  • -Reasoning: accuracy −4.2%
  • -Knowledge: accuracy −9.9%, mIoU −8.5%
  • Latency: full turn 24.07 ± 3.57 s without reasoning; 66.50 ± 11.12 s with NV-Reason-CXR-3B

Limitations

  • Single-participant preliminary evaluation (N=1); no statistical significance or generalizability
  • NV-Reason-CXR-3B local inference is slow (~42 s/call), requiring 8× RTX 8000 48GB GPUs
  • Depends on proprietary GPT-4 API for assessment and response stages
  • Requires reliable eye-tracking hardware for gaze analytics
  • No longitudinal study of knowledge retention

Relevance to Agentic AI / LLM Agents

This paper is a concrete instantiation of multi-agent orchestration (via AutoGen) for a high-stakes domain task, demonstrating how specialized agents with distinct roles (assessment, Socratic coaching, knowledge retrieval, VLM reasoning, safety filtering) can be composed into a coherent, stateful workflow. The BKT-driven conditional routing pattern—where mastery state determines which downstream agents are invoked—illustrates principled agent selection beyond static pipelines. The safety architecture (sanitization + progressive disclosure) is directly relevant to the broader challenge of controlling information leakage in agentic systems with access to privileged knowledge. IMACT-CXR serves as a domain-specific proof-of-concept showing that multi-agent orchestration outperforms single-agent VLMs when tasks require multimodal state tracking and adaptive behavior.