Towards Autonomous and Auditable Medical Imaging Model Development

🕒 Published (v1): 2026-07-12 00:54 UTC · Source: Arxiv · link

Why this paper was selected

Autonomous MLE agent for medical imaging; concrete harness applied to buildable medical AI pipeline

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AMID is an autonomous multi-agent framework that automates medical imaging model development end-to-end, from data profiling through validated artifact delivery. It introduces Data-Conditioned Method Planning (DCMP) to convert task-specific data evidence into executable, parallelizable method lanes, and Verification-Guided Two-Stage Optimization to explore then exploit those lanes while enforcing strict artifact and protocol checks. Evaluated on 20 medical imaging challenge tasks, it outperforms general-purpose MLE agents and approaches human-designed solutions on several tasks.

Problem

Generic autonomous MLE agents fail on medical imaging for two reasons: (1) their search spaces are too coarse-grained — the same nominal task type (segmentation, detection) requires radically different pipeline choices depending on modality, voxel geometry, annotation semantics, and submission format, so generic agents waste compute on medically inappropriate or non-executable pipelines; (2) experimental feedback is expensive, delayed, and unreliable — small cohorts, patient-level splits, scanner/site effects, and metric mismatches mean a single validation score can be misleading, and agents may exploit low-cost shortcuts (incomplete training, leaky splits, invalid artifacts) that produce inflated but untrustworthy evidence.

Method

Data-Conditioned Method Planning (DCMP) operates in two phases. First, a task and data profiler inspects raw files, labels, metadata, and submission specs to produce a planning profile covering image geometry (shape, spacing, anisotropy), supervision type (class balance, foreground ratio, object-size distributions), patient-level leakage risks, metric direction, and submission schema — without training any model. Second, a resource-grounded method search converts these data conditions into a portfolio of method lanes, each grounded in verifiable executable resources: a pre-seeded library (nnU-Net, MONAI) and a curated foundation-model registry (MedSAM, TotalSegmentator, VISTA3D, domain-specific models). DCMP outputs a hybrid method graph where nodes are data conditions/method families/implementation resources and edges encode why each method is plausible under the observed constraints. Each lane specifies a modeling hypothesis, required resources, preprocessing assumptions, validation obligations, approximate training budget, expected artifacts, and likely failure modes.

Verification-Guided Two-Stage Optimization runs over this lane portfolio. Stage 1 (behavior-gated exploration) distributes worker agents across method families; each agent produces an attempt (code commit + predictions + validation score + artifacts), which is frozen and reviewed by an independent reviewer process outside the worker session. The reviewer checks validation split legality, metric direction, artifact completeness, non-duplication, and code-to-artifact traceability. Only reviewer-accepted attempts count as real lane coverage. Once lane coverage reaches plateau, the lifecycle manager promotes the top-\(K\) candidates. Stage 2 (selective exploitation) concentrates workers on promoted candidates for multi-fold cross-validation, ensembling, test-time augmentation, and artifact finalization; the same reviewer gate stays active throughout.

Self-Organizing Agent Loop: a lifecycle manager orchestrates workers running in isolated git worktrees against a shared artifact memory (attempts, notes, scores, checkpoints). It controls worker creation, context injection, interruption, reassignment, and retirement. The framework is backend-agnostic — workers may be Codex, Claude Code, OpenCode, Cursor Agent, or Kiro instances within the same run.

Key Contributions

• First autonomous multi-agent MLE framework targeting general medical imaging model development, producing auditable packages (executable code, validation evidence, submission-ready artifacts) across heterogeneous tasks and modalities.
• DCMP: converts raw data evidence and executable resource inventories into implementation-backed, parallelizable method lanes, preventing generic agents from wasting compute on medically inappropriate or non-runnable pipelines.
• Verification-Guided Two-Stage Optimization: decouples evidence generation (workers) from evidence certification (independent reviewer), enforcing protocol integrity before any attempt can influence lane promotion or final selection.
• Backend-agnostic self-organizing agent loop with a lifecycle manager that handles worker failure, stalling, context pressure, and reassignment without manual intervention.

Results

• Evaluated on 20 ReX-MLE medical imaging challenge tasks spanning detection and segmentation modalities.
• AMID (Codex backend) outperforms all evaluated general-purpose MLE baselines: AIDE, ML-Master, and R&D-Agent across average metric score.
• On several tasks, AMID approaches or matches the "human level" (strong human-designed challenge solutions).
• Category-level bar chart shows AMID leads in Detection, Segmentation, and Average metric scores relative to all three baselines; exact numerical values are not reported in the extracted text beyond the figure.
• Note: this is a preliminary technical report; complete ablation numbers are not provided in the available text.

Limitations

• Preliminary technical report — results and system details may change in the final version.
• Exact per-task performance numbers are not tabulated in the available text; comparisons rely on bar charts with imprecise readable values.
• The pre-seeded resource library (nnU-Net, MONAI) and foundation-model registry require curation and maintenance; gaps in registry coverage could leave valid methods unexplored.
• Fixed-time-budget scheduling policy is described conceptually but not evaluated separately.
• Reviewer gate correctness depends on the reviewer agent's ability to detect subtle leakage, metric mismatch, or invalid artifacts — its own error rate is not characterized.
• No ablation isolating the contribution of DCMP vs. the reviewer gate vs. the two-stage structure.

Relevance to Harnesses / Meta-Harnesses

AMID is a domain-specialized meta-harness: it wraps multiple heterogeneous coding-agent backends (Codex, Claude Code, OpenCode, Cursor Agent, Kiro) under a unified lifecycle manager that enforces shared state, verification gates, promote/retire transitions, and artifact contracts — the defining structure of a meta-harness. The reviewer-as-gatekeeper pattern (an independent agent that certifies evidence before it can influence the main loop) is a reusable meta-harness primitive for any domain where experimental feedback is noisy or expensive. The two-stage explore-then-exploit orchestration and the behavior-gated promotion logic are concrete designs for how a meta-harness can manage a dynamic pool of worker agents without human checkpointing.