Skip to content

An Artifact-based Agent Framework for Adaptive and Reproducible Medical Image Processing

🕒 Published (v1): 2026-03-31 19:28 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces an artifact-based agent framework that wraps a deterministic workflow engine (Snakemake) with an LLM-driven semantic layer to enable adaptive, dataset-aware configuration of medical image processing pipelines while preserving full reproducibility. The agent synthesizes workflow configurations from a modular rule library conditioned on dataset inspection and user goals, then delegates execution to the deterministic engine. Evaluation across three CT/MRI clinical cohorts shows 100% DAG reproducibility and >85% initial rule matching against expert-defined pipelines.

Problem

Real-world clinical deployment of medical image processing requires handling heterogeneous PACS data (mixed modalities, non-standard directory structures, nested DICOM archives) while complying with privacy regulations that preclude cloud processing. Existing workflow engines (Snakemake, Nextflow) guarantee reproducibility only for pre-specified configurations; they provide no mechanism for dataset-aware adaptive configuration or semantic inspection of intermediate workflow state without re-running the pipeline.

Method

The framework introduces an artifact contract that formalizes every workflow output as a tuple \(a_j = (t_j, \phi_j, \psi_j)\), where \(t_j\) is artifact type, \(\phi_j\) are structured attributes (modality, resolution, scanner, measurements), and \(\psi_j\) encodes provenance (generating rule, dependency chain). Two constrained agentic operators act over this artifact registry:

  • Workflow Assembly \(\Gamma_\text{assemble}(g, A, S) = C\): the agent inspects the dataset, iteratively refines the analytical goal \(g\) through dialogue, then selects and composes rules from the modular rule library \(S = \{s_k\}\) whose declared I/O signatures are satisfied by existing artifacts, producing a configuration \(C = (\pi, \theta)\) presented as a DAG for user approval.
  • Semantic Query \(\Gamma_\text{query}(q, A) = r\): natural-language queries are translated into filters over \(\phi_j\) and \(\psi_j\) fields; the LLM never accesses raw files, only the structured artifact registry.

Execution is delegated entirely to Snakemake, which constructs the dependency DAG and ensures deterministic, file-tracked execution. The entire system runs locally via Ollama with DeepSeek-R1 (14B) to satisfy privacy constraints.

Key Contributions

  • Artifact contract formalism: a machine-readable, provenance-aware state representation \(a_j = (t_j, \phi_j, \psi_j)\) covering all intermediate and final workflow outputs.
  • Constrained agentic layer: goal-conditioned workflow assembly and artifact-grounded semantic querying without allowing unconstrained tool invocation that would break reproducibility.
  • Separation of semantic reasoning from deterministic execution: the LLM only plans and queries; Snakemake owns all computation, preserving DAG determinism.
  • Empirical validation on three heterogeneous real-world cohorts (NLST, LungCaTrial, BrainICU) across 9 distinct analytical goals.

Results

  • Reproducibility: DAG equivalence = 100% across all 9 workflow configurations on repeated independent runs after memory clearing.
  • Adaptability — Initial Rule Matching (IRM): >85% overlap with expert-defined configurations across all cohorts/goals; range 85.7%–100%.
  • Adaptability — Planning Iterations (PL): 1–8 user-agent dialogue turns to finalize a configuration.
  • Adaptability — Final Output (FO): 90.7%–100% of sessions produce outputs matching manually curated ground truth.
  • Semantic query accuracy (with artifact contract, DeepSeek-R1 14B): 100% on status and provenance categories; 90–95% on filter/counting queries.
  • Ablation (no artifact contract, filename-only): filter/counting accuracy drops to 10–20%; provenance accuracy drops to 75–80%.

Limitations

  • Semantic query accuracy is bounded by upstream DICOM metadata quality; missing or corrupted headers propagate into inaccurate artifact attributes \(\phi_j\).
  • Adaptability is bounded by rule catalog coverage; analytical objectives not represented in \(S\) require manual rule authoring.
  • Evaluation uses exact-match accuracy over only 20 queries per category — a small sample.
  • Planning iteration count (PL) varies widely (1–8), suggesting inconsistent goal disambiguation behavior across cohort types.
  • Experiments use a single local 14B model (DeepSeek-R1); behavior with smaller or larger models is uncharacterized.

Relevance to Harnesses / Meta-Harnesses

This work is a direct instantiation of a domain-specific meta-harness: a structured orchestration layer that wraps a deterministic execution engine (Snakemake) with an LLM planner, enforcing that the agent operates only over a formally defined state representation (the artifact contract) rather than invoking tools freely. The artifact contract pattern — formalizing intermediate harness state as queryable, provenance-tagged records — is a transferable design principle for any meta-harness that must balance LLM-driven adaptability with reproducibility guarantees. The constrained agentic loop (plan over defined components → delegate execution → update artifact state) directly addresses a core meta-harness challenge: how to give an LLM orchestrator enough flexibility to handle heterogeneous inputs without undermining the determinism required for auditable pipelines.