Skip to content

Flow: Modularized Agentic Workflow Automation

🕒 Published (v1): 2025-01-01 · Source: ICLR · Venue: ICLR 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Flow is a multi-agent framework that represents the agentic workflow as an Activity-on-Vertex (AOV) DAG, enabling both parallel subtask execution and continuous runtime refinement when subtasks fail. It selects among \(K\) LLM-generated workflow candidates by maximizing a parallelism score \(P_{avg}\) and minimizing dependency complexity \(C_{dependency}\), then re-runs this selection loop dynamically throughout execution.

Problem

Existing multi-agent frameworks (MetaGPT, CAMEL, AutoGen) impose static or strictly sequential workflows that cannot adapt to unforeseen failures, reassign agents, or restructure subtask dependencies during execution. Dynamic subtask reallocation in multi-agent settings has not been systematically addressed, causing brittleness when plans meet real-world obstacles.

Method

Workflow representation. The workflow is formalized as an AOV graph \(G = (V, E, A)\) — a DAG where vertices \(v \in V\) are subtasks, directed edges \(e_{ij}\) encode precedence, and \(A\) is the agent set. Each subtask carries a dictionary record: {subtask_requirement, status, data, num_parents_not_completed, child, agent}, serializable as JSON.

Modular workflow selection. An LLM generates \(K\) candidate AOV graphs from an initialization prompt. Each candidate is scored on two metrics:

\[P_{avg} = \frac{1}{T}\sum_{t=1}^{T}|S_t|\]

(average concurrent subtasks per step) and

\[C_{dependency} = \sigma_{\deg(v_i)} = \sqrt{\frac{1}{|V|}\sum_{v_i \in V}(\deg(v_i) - \bar{d})^2}\]

(std. dev. of node degree). The candidate with highest \(P_{avg}\) — ties broken by lowest \(C_{dependency}\) — is selected. Theorem 1 formally proves that additional dependencies strictly reduce expected subtask completion count, justifying this selection criterion.

Runtime refinement. A global LLM inspector continuously monitors agent outputs and statuses. On subtask failure, \(K\) new candidate AOVs are generated conditioned on the current graph \(G^t\) and all execution logs \(D^t\); the same selection rule applies. Allowed update operations include deletion, addition, editing, rerunning, and agent reassignment, all localized to the failing module.

Execution. A topological sort of the selected AOV produces a step sequence; subtasks with num_parents_not_completed == 0 execute concurrently. When two subtasks require the same agent in the same step, a clone \(a'_j\) is created.

Key Contributions

  • Formal modularity metrics (\(P_{avg}\), \(C_{dependency}\)) for scoring and selecting LLM-generated workflow DAGs, with a proof that denser dependency graphs reduce expected task success.
  • A runtime workflow-refinement loop that allows localized, LLM-driven restructuring of the AOV graph without cascading disruption to unaffected subtasks.
  • A dictionary/JSON-based harness implementation that is directly LLM-readable, enabling the global inspector to reason over workflow state without parsing overhead.
  • Empirical evaluation on three open-ended coding tasks against AutoGen, CAMEL, and MetaGPT.

Results

All experiments use GPT-4o-mini and GPT-3.5-Turbo agents; 5 trials per framework per task; 50 human raters.

  • Gobang game development: Flow 100% overall success / 4.0 human rating vs. AutoGen 60%/2.26, MetaGPT 73%/1.24, CAMEL 27%/2.50.
  • LaTeX Beamer writing: Flow 100%/3.33 vs. AutoGen 67%/3.00, MetaGPT 60%/1.83, CAMEL 66%/1.83.
  • Website design: Flow 80%/3.28 vs. AutoGen 73%/2.62, MetaGPT 80%/1.72, CAMEL 53%/2.02.
  • Average across tasks: Flow 93% success, 3.54/4 human satisfaction vs. best competitor AutoGen at 66.7%/2.63.

Limitations

  • Evaluation restricted to coding/writing tasks; authors acknowledge this may bias results toward frameworks optimized for code generation (MetaGPT).
  • Only 5 trials per condition — variance estimates are unreliable.
  • The RL fine-tuning extension (reward on completion speed, resource use) is described but not implemented.
  • Global LLM inspector adds latency and cost on every subtask completion; no ablation on inspector call frequency or cost.
  • Only GPT-4o-mini and GPT-3.5-Turbo tested; generalizability to other LLM backends unknown.

Relevance to Harnesses / Meta-Harnesses

Flow is directly a meta-harness: it dynamically constructs, evaluates, and rewires the execution graph that governs sub-agent pipelines — the exact function a meta-harness performs over lower-level agents or tools. The candidate-generation-and-scoring loop (\(K\) AOVs ranked by \(P_{avg}\) / \(C_{dependency}\)) is a concrete template for harness self-optimization, analogous to how a meta-harness might select among alternative pipeline topologies. The dictionary/JSON workflow representation — where the structure itself is LLM-readable and mutable — is a practical pattern for harnesses that need to introspect and rewrite their own orchestration logic at runtime. The modularity theorem provides a formal grounding for why harnesses should prefer loosely coupled sub-pipelines over tightly chained ones.