Skip to content

A Self-Evolving Agentic System for Automated Generation and Execution of Biological Protocols

🕒 Published (v1): 2026-06-30 14:51 UTC · Source: Arxiv · link

Why this paper was selected

Self-evolving agentic system with SOP-to-code alignment and feedback loops; generalizable scaffold

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ProtoPilot is a self-evolving multi-agent system that converts free-form natural-language experimental objectives into validated SOPs, SDK-compliant instrument code, and physically executed wet-lab workflows, then incorporates execution feedback into subsequent iterations. It ships alongside BioLab bench, a 294-task benchmark spanning synthetic and molecular biology, with rubric-based validators and real wet-lab tests. On that benchmark it achieves a 96.6% gate pass rate and 90.2% Top@3 expert preference, substantially outperforming all baselines.

Problem

Existing AI systems for biology stop at "plausible protocol text." The handoff from biological intent to physical experimentation is fragile: scientific rationale, sample states, quantitative parameters, device constraints, and QC readouts must remain mutually consistent as a goal passes through five successive representations (intent → protocol → manual SOP → device SOP → instrument code). Misalignment at any layer produces workflows that cannot be executed, reproduced, or repaired after failure. No prior benchmark evaluated this full alignment chain with execution-level validity gates and real wet-lab tests.

Method

ProtoPilot implements two interlocking mechanisms inside a stateful closed loop:

1. Hierarchical coordinator–worker SOP construction. An Orchestrator Agent decomposes the user objective into modules, then issues structured task packets (module goal, accepted upstream state, downstream dependencies, device constraints) to a Protocol Expert Agent trained with reinforcement learning. The expert returns step-level procedures and parameters; rubric-based validators (biological plausibility, completeness, parameter consistency, safety, SDK alignment) gate each module before the Orchestrator integrates it into the evolving SOP and plans the next module.

2. Self-evolving skill library for SOP-to-code conversion. A Coding Agent maintains three hierarchical skill classes: L1 parameter-confirmation skills (resolving qualitative SOP descriptions into executable values), L2 deck/layout planning skills (slot assignment, collision avoidance, well maps), and L3 instrument-operation/code-specification skills (SDK calls, module state, liquid handling). Skills are initialized from expert rules and iteratively improved through create→evaluate→revise loops using rubric validators, code execution checks, and wet-lab feedback—encoding stable operational principles reusable across tasks and device platforms.

A multilayer agentic memory bank (domain knowledge base, per-task working memory, and a task-experience base of completed trajectories and feedback) supports planning and consistency checking throughout. Human-in-the-loop checkpoints can be injected at material availability, instrument selection, and adjustment decision points.

Key Contributions

  • ProtoPilot: end-to-end closed-loop multi-agent system from natural-language intent to physically executed and feedback-revised wet-lab workflows
  • BioLab bench: 294 tasks derived from 98 gold-standard protocols at three difficulty tiers (L1–L3), with a seven-dimension rubric, device-level gate checks, and real experimental validation
  • Self-evolving skill library: runtime-updated, three-tier hierarchical skills that transfer across device platforms (MGI AlphaTool, Hamilton STAR, Opentrons OT-2, Tecan EVO)
  • Layer-wise rubric validators as internal gatekeepers enforcing cross-layer alignment at each representation transition
  • Wet-lab validation across four workflows of increasing complexity, including feedback-corrected PCA-based de novo DNA assembly

Results

  • Protocol quality: rubric score 94.7 ± 0.4, beating BioProAgent (next-best) by 3.2 points and all general-purpose LLMs by >6 points across all difficulty tiers
  • Blind expert preference: Top@1 70.6 ± 16.4%, Top@3 90.2 ± 9.8%; MRR 0.82 ± 0.12 vs. second-best LabScript-AI at 0.37 ± 0.07
  • Protocol-to-code gate pass rate: 96.6% overall (93.3% L1 / 97.0% L2 / 96.3% L3), vs. max 35.0% for general-purpose LLMs and 64.6% for LabScript-AI
  • Cross-device generalization: gate pass rates 91.2% / 85.3% / 88.2% / 91.2% on four platforms (spread 5.9 pp); LabScript-AI spread was 47.1 pp
  • Opentrons OT-2 vs. purpose-built baseline: ProtoPilot 88.2% vs. OpenTrons-AI 32.4%
  • Wet-lab: 24/24 positive colony PCR clones; 3/3 Sanger-confirmed WT plasmids; 7/8 GLuc + 8/8 RLuc mutants sequence-confirmed; feedback-guided PCA assembly produced sequence-confirmed EGFP and A501_Ec.CPS-derived targets (93/96 PCR-positive, all Sanger-confirmed after protocol revision)
  • Spearman \(\rho = 0.929\) (\(p = 0.003\)) between automated rubric scores and expert MRR across 7 systems, validating rubric as a reliable proxy

Limitations

  • Blind expert evaluation used only 34 queries rated by 3 scientists; Kendall's \(W = 0.495\) reflects moderate inter-rater agreement, indicating inherent subjectivity in protocol ranking
  • One GLuc mutant (1/8) was not sequence-confirmed, and one vector amplification/transformation cycle failed before feedback-guided correction, indicating residual failure modes in complex multi-step assemblies
  • The self-evolving skill library requires domain-specific initialization from expert rules and SDK documentation; out-of-domain instruments or novel chemistries may degrade generalization until sufficient trajectories accumulate
  • Benchmark tasks are derived from 98 gold-standard protocols within synthetic and molecular biology; coverage of other wet-lab domains (e.g., cell culture, proteomics, imaging) is not demonstrated
  • The paper text provided is truncated; additional limitations stated in the full paper may not be reflected here

Relevance to Harnesses / Meta-Harnesses

ProtoPilot is a direct instantiation of a domain-specific meta-harness: it orchestrates multiple specialized sub-agents (Orchestrator, Protocol Expert, Coding Agent) through structured task packets and rubric-based gates, analogous to a harness's role of dispatching, validating, and routing between tools or workers. The self-evolving skill library—where skills are created, evaluated against validators, and revised in a runtime loop—is a concrete implementation of a harness that accumulates reusable operational knowledge rather than re-prompting from scratch each run. The five-layer representation with stage-wise guards mirrors the harness pattern of enforcing interface contracts between pipeline stages. For researchers tracking harnesses and meta-harnesses, ProtoPilot demonstrates how a closed-loop, feedback-driven orchestration architecture can be grounded in physical execution constraints—extending the harness paradigm from purely digital workflows into real-world cyber-physical systems.