Skip to content

\(A^2Flow:\) Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators

🕒 Published (v1): 2025-11-23 07:32 UTC · Source: Arxiv · Venue: AAAI · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

\(A^2Flow\) eliminates the hand-crafted operator bottleneck in automated agentic workflow generation by extracting reusable, task-agnostic execution operators directly from expert demonstrations via a three-stage LLM-driven abstraction pipeline. It then searches for optimal workflows over these auto-derived operators using MCTS, augmented by an operator memory mechanism that accumulates intermediate outputs across nodes. On eight benchmarks spanning five domains, it surpasses prior MCTS-based workflow optimizers by 2.4% on standard tasks and 19.3% on embodied/game tasks while cutting resource usage by 37%.

Problem

Existing automated workflow frameworks (AFlow, DebFlow, MermaidFlow) still rely on manually predefined, task-specific operators—bundles of LLM-invoking nodes and edges—that must be redesigned for each new domain. This prevents full automation, degrades generalization to open-world tasks (embodied control, WebShop), and introduces human-design bias into the MCTS search space.

Method

Self-Adaptive Abstraction Operators (three-stage extraction):

  1. Case-based Initial Operator Generation — for each case \(C_i\) in a 20% validation split, an LLM produces case-aware operators: \(O^{(e)} = \{o_{i,j} = \mathcal{E}(C_i, P_e, M)\}\). Each operator is expressed as a single-input/single-output Python code block.

  2. Operator Clustering and Preliminary Abstraction — an LLM clusters \(O^{(e)}\) by functional similarity to yield preliminary abstractions \(O^{(a)} = \mathcal{C}(O^{(e)}, P_a, M)\).

  3. Deep Extraction for Abstract Execution Operators — \(m=6\) independent reasoning paths are generated by varying temperature; each path undergoes three rounds of iterative Long-CoT refinement (providing prior outputs \((I, P_t, o_1, \ldots)\) at each step). An LLM aggregation function merges results across paths into the final task-aware operator set \(O^{(t)}\).

A Python executor validates syntax/executability after each step and triggers LLM reflection on failure.

Operators Memory Mechanism — each node \(k\) accumulates all prior operator outputs: \(M_k = M_{k-1} \cup \{o_k\}\), and computes \(o_k = f_k(\text{input}_k, P_k, M_{k-1})\), breaking the strictly sequential dependency of vanilla AFlow.

Workflow search follows AFlow's MCTS loop (initialize → select → expand → evaluate → backpropagate), but the expansion phase draws from \(O^{(t)}\) and \(M\) rather than predefined operators.

Key Contributions

  • Fully automated extraction of abstract execution operators from raw expert data, removing all manual operator predefinition.
  • Three-stage operator pipeline: case-level extraction → functional clustering → multi-path Long-CoT refinement with self-consistency aggregation.
  • Operators Memory Mechanism enabling cross-step context accumulation within each workflow node.
  • Demonstrated generalization to embodied (ALFWorld) and game (TextCraft) domains—tasks where prior predefined-operator frameworks fail.
  • Pareto-optimal cost-performance: auto-discovered workflows allow weaker LLMs (GPT-4o-mini) to match or exceed stronger ones.

Results

  • Standard benchmarks (reading comprehension, coding, math reasoning; 6 datasets): +2.4% average over AFlow; DROP +4.5%, MATH +4.1% over AFlow using GPT-4o-mini as executor.
  • Embodied/game benchmarks (ALFWorld seen/unseen + TextCraft): +19.3% average over AFlow using DeepSeek-v3 as executor; ALFWorld seen 25.0 vs 17.1, TextCraft 59.0 vs 53.0.
  • Resource reduction: 37% lower token cost vs. state-of-the-art baselines.
  • Ablation (on a composite score baseline of 56.2): removing initial operators −11.7%, removing deep extraction −8.2%, removing operator clustering −3.0%, removing operator memory −4.1%.
  • HumanEval is the one exception where gains are modest, attributed to predefined Python-interpreter operators providing a strong baseline.

Limitations

  • Operator extraction requires a labeled expert demonstration set per task type; the framework is not zero-shot in new domains.
  • The three-stage extraction adds upfront LLM inference cost before any workflow search begins, which is not quantified in isolation.
  • Reflection/retry loops for code validity can cascade into extended extraction times; worst-case depth is not analyzed.
  • HumanEval performance lags when predefined Python-interpreter operators are available to baselines, suggesting abstract operators alone cannot yet match hand-tuned tool integrations.
  • Evaluation uses a fixed 20%/80% validation/test split with seed 42; sensitivity to this split is not reported.

Relevance to Harnesses / Meta-Harnesses

\(A^2Flow\) is directly a meta-harness for workflow generation: rather than executing a fixed pipeline, it dynamically synthesizes the building blocks (operators) of other harnesses from data, then orchestrates MCTS search over those blocks. The operator memory mechanism is an architectural pattern relevant to any harness that chains LLM calls and needs to propagate intermediate state across steps without full context re-injection. The ablation decomposing extraction stages offers a reusable blueprint for harness designers who want to bootstrap operator vocabularies from demonstrations rather than manual specification. The cost-performance Pareto analysis—showing that auto-designed workflows with cheaper executors can outperform expensive hand-crafted ones—is directly actionable for anyone building scalable multi-agent harnesses.