\(A^2Flow:\) Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators

🕒 Published (v1): 2025-11-23 07:32 UTC · Source: Arxiv · Venue: AAAI · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

A²Flow is a fully automated framework for agentic workflow generation that eliminates manually predefined operators by deriving reusable "Self-Adaptive Abstraction Operators" from raw expert demonstrations via a three-stage LLM-driven extraction pipeline. It augments MCTS-based workflow search with an Operator Memory Mechanism that accumulates intermediate outputs for richer node-level context. On general benchmarks it improves over AFLOW by 2.4% average and 19.3% on embodied/game tasks while reducing token cost by 37%.

Problem

Prior automated workflow generation methods (AFlow, DebFlow, MermaidFlow) still depend on manually predefined, task-specific operators. This dependency (1) limits generalization to open-world and embodied tasks unseen during operator design, (2) introduces human bias into the search space, and (3) creates unnecessary overhead from redundant or suboptimal operators that are never empirically validated for optimality.

Method

A²Flow replaces hand-crafted operators with a three-stage automatic extraction pipeline applied to a 20% validation split of expert demonstration data:

1. Case-based Initial Operator Generation: An LLM prompted with case-specific examples generates per-instance code-based operators (e.g., ObserveEnvironment(), CreatePlan()), each structured as a single-input/single-output Python block.
2. Operator Clustering and Preliminary Abstraction: An LLM-guided functional clustering step groups semantically similar operators across cases into coarser preliminary abstractions (e.g., TaskPlanner(), Locator(), Validator()).
3. Deep Extraction for Abstract Execution Operators: Six parallel reasoning paths are generated by varying LLM temperature; each path undergoes three iterative CoT-prompted refinement steps producing increasingly abstract operators. An LLM aggregation function merges all paths into a compact final set (e.g., Planner(), Executor(), Validator()). A Python executor provides syntax feedback, triggering reflection-based regeneration on failure.

These abstract operators replace AFlow's predefined operator set in an MCTS-based workflow search. The Operator Memory Mechanism augments each node: rather than depending only on the immediately preceding operator's output, each operator k receives a running memory M_{k-1} containing all prior outputs, which is updated after each step: M_k = M_{k-1} ∪ {o_k}.

Key Contributions

• Three-stage self-adaptive abstraction pipeline that derives compact, generalizable operators from expert data without manual specification.
• Operator Memory Mechanism enabling cross-operator context sharing within workflow execution, improving node-level decision quality.
• Demonstrated generalization to embodied (ALFWorld) and game (TextCraft) task domains where prior predefined-operator methods fail or significantly underperform.
• Pareto-front analysis showing A²Flow workflows enable cheaper executor models (GPT-4o-mini) to surpass stronger models (GPT-4o) at lower cost.

Results

General benchmarks (Table 1, executor: GPT-4o-mini, vs. AFLOW): - DROP (reading comprehension F1): 85.1 vs. 80.6 (+4.5 pp) - MATH Lv5 (solve rate): 58.5% vs. 56.2% (+4.1 pp over full AFLOW; +2.3 pp avg over all predefined-operator methods) - HumanEval (pass@1): 92.4% vs. 90.9% (+1.5 pp) - MBPP (pass@1): 85.0% vs. 83.4% (+1.6 pp) - GSM8K: 93.8% vs. 93.5% (+0.3 pp) - HotpotQA F1: 74.1% vs. 73.5% (+0.6 pp) - Average over 6 general tasks: 81.5% vs. 79.6% (+2.4 pp)

Embodied/game benchmarks (Table 2, executor: DeepSeek-v3, vs. AFLOW): - ALFWorld Seen: 25.0 vs. 17.1 (+7.9 pp) - ALFWorld Unseen: 31.3 vs. 26.6 (+4.7 pp) - TextCraft: 59.0 vs. 53.0 (+6.0 pp) - Average: 38.4 vs. 32.2 (+19.3% improvement)

Ablation (MATH Lv5, Table 3): - Full model: 58.5%; removing Operators Memory: 53.9% (−4.1 pp); removing all Self-Adaptive Abstraction Operators: 49.6% (−11.7 pp from full model, −8.9 pp below AFLOW baseline of 56.2%)

Cost: A²Flow reduces resource usage by 37% over AFLOW; discovered workflows allow GPT-4o-mini to dominate GPT-4o on the DROP cost-performance Pareto front.

Limitations

• Operator extraction relies on a 20% validation split of task-specific expert demonstration data; tasks with no such data are not addressed.
• On HumanEval, A²Flow does not achieve the top result among all methods; predefined operators that include a Python interpreter invocation remain a strong baseline for code tasks.
• On GSM8K, gains are marginal (+0.3 pp) because baseline performance is already near ceiling.
• The embodied/game evaluation uses only ALFWorld and TextCraft; broader embodied domains and long-horizon tasks remain underexplored.
• No analysis of how operator quality degrades with fewer or noisier expert demonstrations.
• The deep extraction stage uses six parallel reasoning paths with three iterative refinements each, implying non-trivial upfront LLM cost during the extraction phase (not quantified separately).

Relevance to Agentic AI / LLM Agents

A²Flow directly targets a core bottleneck in agentic system design: the manual engineering of workflow operators, which currently limits scalability and domain transfer. By showing that operators themselves can be discovered automatically from data rather than hand-specified, it advances the line of work on meta-level automation of agent architectures (alongside AFlow, ADAS, DSPy). The Operator Memory Mechanism is independently relevant as a lightweight form of episodic memory within structured multi-step execution, complementing longer-horizon memory approaches. The strong generalization to embodied tasks (ALFWorld +19.3% over AFLOW) is particularly significant because prior automated workflow methods consistently failed on open-world domains, suggesting that operator abstraction is a key missing ingredient for generalizable LLM agents.