Towards Autonomous and Auditable Medical Imaging Model Development¶

🕒 Published (v1): 2026-07-12 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

LLM agents autonomously run medical imaging ML pipelines; directly relevant to applied medical AI + agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

AMID is an autonomous multi-agent framework that turns medical imaging challenge tasks into auditable model packages by combining data-conditioned method planning with a two-stage, reviewer-gated optimization loop. It outperforms general-purpose MLE agents across 20 heterogeneous medical imaging tasks and approaches human-level challenge solutions on several. The system is backend-agnostic, supporting Codex, Claude Code, OpenCode, Cursor Agent, and Kiro as coding-agent backends.

Problem¶

Generic autonomous MLE agents (AIDE, ML-Master, R&D-Agent) fail to transfer to medical imaging for two structural reasons: (1) their search spaces are too coarse—appropriate method families depend on modality, voxel spacing, annotation semantics, patient-level splits, and submission format, not just the nominal task type (classification/segmentation/detection); (2) experimental feedback is expensive, delayed, and unreliable—a single validation score can be corrupted by leakage, metric mismatch, fold errors, or invalid prediction artifacts, yet generic agents use unverified scores to drive search. Medical imaging also requires domain-conditioned engineering that foundation models (MedSAM, TotalSegmentator, BioMedParse) alone cannot provide.

Method¶

AMID decomposes development into three tightly coupled components:

Data-Conditioned Method Planning (DCMP). Given a task definition and dataset, AMID first profiles the data (modality, dimensionality, voxel spacing, label statistics, leakage risks, submission schema) without training any model. It then performs resource-grounded method search against a pre-seeded local library (nnU-Net, MONAI) and a curated foundation-model registry (MedSAM, MedSAM2, VISTA3D, TotalSegmentator, domain-specific models), checking each candidate for executability and data compatibility. The output is a hybrid method graph whose leaves are method lanes—each specifying a modeling hypothesis, data conditions, required resources, preprocessing assumptions, validation obligations, training budget, expected artifacts, and failure modes. Fallback and paper-only lanes are downranked rather than silently removed.

Verification-Guided Two-Stage Optimization. - Stage 1 (Behavior-gated Exploration): Worker agents run in parallel across method families and submit attempts tied to specific lanes. An independent reviewer—decoupled from the workers—evaluates each frozen commit, checking validation split legality, metric direction, complete prediction files, non-duplicate payloads, and traceability from code to artifacts. Only reviewer-accepted attempts count toward lane coverage and promotion decisions. - Stage 2 (Selective Exploitation): Promoted candidates receive focused compute from optimizer, challenger, repair, and finalizer workers. The same reviewer gate remains active; only artifacts satisfying the full validation protocol enter the accepted candidate set.

Self-Organizing Agent Loop. A central lifecycle manager maintains explicit per-worker state (initialization → editing → evaluation → reflection → reassignment/retirement). Workers run in isolated git worktrees but share a common task contract, method lanes, attempt records, skills, and audit artifacts. The manager can interrupt and resume sessions with targeted prompts rather than cold-restarting, enabling recovery from stalls, context exhaustion, and implementation failures.

Key Contributions¶

First autonomous multi-agent MLE framework targeting general medical imaging model development, producing auditable packages with executable code, validation evidence, and submission-ready prediction artifacts across heterogeneous tasks and modalities.
Data-Conditioned Method Planning (DCMP): conditions search space construction on actual data profiles and verified executable resources, replacing generic pipeline search with implementation-backed method lanes.
Verification-Guided Two-Stage Optimization: separates evidence admissibility (reviewer) from evidence generation (workers) to prevent unverified scores from corrupting the search trajectory.
Backend-agnostic execution substrate enabling mixed-backend runs (Codex, Claude Code, OpenCode, Cursor Agent, Kiro) sharing the same shared artifact memory and task contract.

Results¶

Evaluated on 20 ReX-MLE medical imaging challenge tasks spanning detection, segmentation, and other modalities/prediction types.
AMID (primary backend: Codex) outperformed all evaluated general-purpose MLE baselines: AIDE, ML-Master, and R&D-Agent, on average metric score across detection and segmentation categories.
On several tasks, AMID approached or matched strong human-designed challenge solutions (denoted "Human level" in Figure 1).
The two-stage optimization curve shows a clear jump in best valid score at the promotion gate, confirming the benefit of selective exploitation over continued broad exploration.

(Note: the paper is a preliminary technical report; per-task numeric scores are shown in figure form only and not tabulated in the provided text.)

Limitations¶

Explicitly a preliminary/ongoing technical report; results and claims are not yet peer-reviewed.
Evaluation is limited to the ReX-MLE benchmark (20 tasks); generalization to clinical deployment settings is not assessed.
High GPU budgets and long training runs required for medical imaging experiments constrain the number of method lanes that can be practically explored per run.
DCMP's resource-grounded search depends on the quality and coverage of the pre-seeded local library and foundation-model registry; missing or gated resources reduce lane quality.
The independent reviewer is rule-based; it may not catch all forms of subtle leakage or metric-gaming that require domain expertise to detect.
No ablation isolating the contribution of DCMP vs. the two-stage optimization vs. the reviewer gate is included in the provided text.

Relevance to Agentic AI / LLM Agents¶

AMID is a concrete instantiation of the agentic MLE paradigm applied to a high-stakes domain, directly addressing the gap between general-purpose MLE agents and domain-constrained engineering tasks. The reviewer-as-gatekeeper design—decoupling evidence generation from evidence certification—is a transferable architectural pattern for any agentic loop where feedback can be unreliable or manipulable. The DCMP module operationalizes a key open problem in agentic AI: how to convert a broad search space into an executable, parallelizable plan grounded in the actual environment rather than task-level priors. The lifecycle manager's explicit state machine for worker sessions (rather than a flat process pool) is also a model for robust multi-agent orchestration under partial failure.