Skip to content

AgentCo-op: Retrieval-Based Synthesis of Interoperable Multi-Agent Workflows

🕒 Published (v1): 2026-05-19 19:22 UTC · Source: Arxiv · link

Why this paper was selected

Retrieval-based synthesis of interoperable multi-agent workflows for open-ended science

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgentCo-op reframes automated multi-agent workflow design as retrieval-based synthesis rather than benchmark-driven search: given a task specification, it retrieves skills, tools, and externally developed agents from libraries, composes them into a typed directed-graph workflow, and applies bounded evidence-guided local repair to failing nodes at runtime. Evaluated on open-world genomics tasks and six standard benchmarks, it achieves best performance on four of six benchmarks while consistently lowering per-task token cost versus multi-agent baselines.

Problem

Existing automated workflow design methods (ADAS, AFlow, AgentSquare) require a curated benchmark \(D\) and a scalar evaluation function: \(W^* = \arg\max_{W \in \mathcal{W}} \text{Eval}(W; D)\). This breaks down in scientific domains where tasks are open-ended, no ground-truth output exists, and "success" depends on heterogeneous evidence (statistical significance, biological plausibility, provenance). A second gap is interoperability: independently developed agents carry incompatible environments and interfaces, so placing them together does not yield coherent execution.

Method

AgentCo-op formalizes a task as \(x = (g, c, r, \Omega)\) (goal, context, constraints, user-provided resources) and maintains a global library \(S\) of typed artifacts (reference resources, skills, tools, external agent repositories). Workflow synthesis produces: $\(W = \text{SYNTHESIZE}(x, S) \triangleq (R, G, \phi, \Pi)\)$ where \(R\) is the set of agent roles, \(G = (V, E)\) is the dependency graph, \(\phi: R \to 2^S\) attaches retrieved artifacts to each role, and \(\Pi\) is the typed interface protocol governing artifact handoffs.

Five pipeline stages: Planning (formulate retrieval plan), Retrieval (fetch skills/tools/repo metadata), Synthesis (build initial graph, wrap external agents in Docker, align I/O via typed artifact schemas), Execution (run graph), Review (monitor logs, errors, cost signals; trigger bounded local repair \(G \to G' = (V', E')\) on implicated nodes only, halting when validation passes, budget is exhausted, or max rounds reached).

External agents and entire GitHub repositories are wrapped as Docker container nodes via a Repo2Run-style build-and-test loop, allowing environment-isolated heterogeneous components to plug into the graph. An existing search-produced workflow (e.g., AFlow output) can be imported as a structural prior in \(\Omega\) and then grounded and repaired.

Key Contributions

  • Formulates multi-agent workflow design as retrieval-based synthesis, removing the dependency on scalar rewards or curated benchmarks.
  • Demonstrates coordination of independently developed scientific agents (TissueAgent + GeneAgent) and domain workflows (Seurat + Signac) via typed artifact handoffs and Docker isolation, without redesigning either repository.
  • Shows synthesis and search are complementary: importing an AFlow-searched MBPP graph and grounding it with retrieved components + local repair further improves it.
  • Achieves best performance on 4/6 standard QA/math/code benchmarks (GPT-4o-mini backbone) while reducing test-time token cost versus multi-agent baselines.

Results

  • Standard benchmarks (6 total): Best on 4/6 (HotpotQA, DROP, HumanEval, GSM8K or MBPP subset—exact four not fully listed) under matched GPT-4o-mini backbone; best average score across all six.
  • Token cost: Consistently lower per-task cost than multi-agent collaboration baselines (MultiPersona, LLM-Debate, Reconcile).
  • Cross-modality marker discovery (PBMC multiome): Combined modality (RNA + ATAC) improves macro precision to 0.303 vs. 0.195 (RNA alone) and 0.110 (ATAC alone) on CellMarker 2.0; recall 0.124 vs. 0.102 (RNA) and 0.061 (ATAC); trend holds per cell type for majority of types.
  • Domain agent coordination (MERFISH heart): Synthesized workflow identified 53 upregulated aFibro markers in AVN/AV ring and correctly characterized them as an AV canal- and node-associated fibroblast program via broker-mediated typed handoff.

Limitations

  • Requires pre-existing curated skill/tool libraries (\(S\)) with typed I/O schemas; library quality and coverage directly bound synthesis quality.
  • Docker containerization adds setup overhead and is fragile if repositories lack tests or clear build specs.
  • Local repair is bounded (budget + round caps); persistent or structural failures in the initial synthesis may not be recoverable.
  • Open-world evaluation lacks quantitative ground truth, making cross-system comparison difficult beyond qualitative demonstration.
  • No ablation fully isolating the contribution of retrieval versus local repair versus typed artifacts.

Relevance to Harnesses / Meta-Harnesses

AgentCo-op is a concrete meta-harness architecture: it does not itself solve tasks but orchestrates heterogeneous task-solving agents, managing their lifecycle (Docker build, I/O alignment, execution monitoring, repair). The five-stage pipeline (Plan → Retrieve → Synthesize → Execute → Review) mirrors the structure of software meta-harnesses that dynamically load and coordinate modules. The "synthesis + search are complementary" finding is important for harness designers—it suggests that a retrieval-based composition layer can augment and improve graphs produced by search-based harnesses like AFlow, implying hybrid meta-harnesses that combine both paradigms. The typed artifact handoff mechanism is a transferable pattern for any harness requiring interoperability across independently developed components with incompatible environments.