DyFlow: Dynamic Workflow Framework for Agentic Reasoning¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

DyFlow is a dynamic workflow framework for LLM-based agents that replaces static, pre-planned execution graphs with a hierarchical designer–executor architecture that continuously revises subgoal plans based on real-time intermediate feedback. A lightweight DyPlanner (Phi-4, 14B) trained via supervised distillation + KTO self-play preference optimization acts as the designer, while any off-the-shelf LLM serves as the executor. Across five reasoning domains, DyFlow achieves an average accuracy of 61.45, outperforming the best prior framework (MaAS at 57.74) by +3.7 points.

Problem¶

Existing LLM agent workflows are either fully static (MetaGPT, AutoGen, CAMEL) or apply workflow optimization offline at the dataset level (AFlow via MCTS, ADAS via meta-search), then execute the resulting plan rigidly without revision. Methods that introduce query-level configurability (DyLAN, MaAS) still do not adapt subgoal decomposition in response to runtime execution signals. This limits robustness when intermediate results are wrong or incomplete, and reduces cross-domain generalization since workflows are tuned to specific distributions.

Method¶

DyFlow models task execution as a dynamic sequence of stage subgraphs \(G_t = (V_t, E_t, v^t_\text{start}, C^t_\text{end})\), where \(V_t\) contains operator instances drawn from a finite template set \(O\).

Designer: A policy \(\pi_\theta\) that, at each step \(t\), receives a GPT-4o-mini-condensed summary of the current state \(s_t\) (including task spec, prior subgraphs, intermediate outputs, errors) and samples the next stage subgraph \(G_t \sim \pi_\theta(\cdot \mid f_\text{summary}(s_t))\). Control flow (branching, loops, backtracking) is handled implicitly by the designer replanning rather than by hardcoded edge logic in the graph.

Executor: Any frozen LLM \(\pi_\text{exec}\) that runs operator instances in topological order, retrieving inputs from and writing outputs to a global key–value memory buffer \(M\).

Designer training — two phases: 1. Supervised distillation (SFT): Fine-tune on successful \((f_\text{summary}(s_t), G^{\text{expert}}_t)\) pairs extracted from GPT-4.1-generated trajectories. 2. Self-play preference optimization (KTO): Collect self-generated trajectories labeled preferred/discarded by full-trajectory success; optimize with KTO loss against the SFT reference policy \(\pi_\text{ref}\). Avoids DPO pairwise ranking (infeasible for multi-step execution) and online RL (sparse/delayed rewards).

Key Contributions¶

DyFlow framework: execution-adaptive, subgoal-level dynamic replanning via a designer–executor hierarchy with stage subgraph formalism.
DyPlanner: a 14B Phi-4-initialized designer trained with SFT distillation + KTO self-play; achieves parity with GPT-4.1 and Claude-3.7-Sonnet designers at fraction of cost.
Cross-domain generalization: DyFlow trained on MATH, PubMedQA, LiveBench generalizes zero-shot to HumanEval (92.07) and SocialMaze (17.18), both held out during training.
Systematic evaluation: five reasoning domains (social, biomedical, math, code, logic), multiple executors (GPT-4o-mini, Phi-4, GPT-4.1-mini), ablations over every core component.

Results¶

Overall average accuracy: DyFlow 61.45 vs. MaAS 57.74 (+3.71), AFlow 57.50 (+3.95), CoT 54.43 (+7.02), Vanilla 53.44 (+8.01). All methods use Phi-4 as executor.
Per-domain (Phi-4 executor):
SocialMaze (zero-shot): 17.18 vs. MaAS 13.36, AFlow 11.45
PubMedQA: 72.91 vs. MaAS 69.32, AFlow 69.72
MATH: 76.40 vs. MaAS 73.60, AFlow 74.00
LiveBench: 48.67 vs. MaAS 44.00, AFlow 43.33
HumanEval (zero-shot): 92.07 vs. MaAS 88.41, AFlow 89.02
Pass@5: DyFlow reaches 0.9817 on HumanEval; consistently wider margin over CoT as k increases.
Cross-executor: DyPlanner improves over CoT by +11.1 on SocialMaze with GPT-4.1-mini executor; gains present across all three executor sizes tested.
Cross-designer (Table 2): DyPlanner (14B) achieves 61.45 avg vs. GPT-4.1 designer 59.83 and Claude-3.7-Sonnet designer 61.16.
Ablation (Table 4): removing dynamic planning gives largest drop (avg 56.48); removing KTO: 57.45; removing SFT: 59.00; removing dynamic operators: 56.97.

Limitations¶

Designer training requires successful expert trajectories generated by a large proprietary model (GPT-4.1), adding an upfront data-collection cost.
The state summarizer is always GPT-4o-mini, introducing a fixed external dependency and latency per step.
Trajectory-level KTO signal cannot distinguish which individual subgraph in a failed trajectory caused the failure, making credit assignment coarse.
Evaluated only on five benchmarks; interaction with tool-use, retrieval, or long-horizon real-world environments is not explored.
Future work on richer error detection and expanded operator sets is acknowledged as outstanding.

Relevance to Agentic AI / LLM Agents¶

DyFlow directly advances the core open problem in agentic AI: moving from static, pre-compiled workflows to genuinely adaptive execution where the agent revises its own decomposition in response to what happens at runtime. Its separation of a trainable designer from a plug-in executor offers a practical architecture for deploying adaptive planning without retraining the base LLM, which is a key scalability concern. The distilled self-play training method provides a concrete recipe for endowing small open-weight models with planning capability comparable to frontier models, lowering the cost of deploying autonomous agents. The zero-shot cross-domain results on SocialMaze and HumanEval strengthen the case that feedback-driven subgoal revision generalizes beyond task-specific workflow optimization, a critical property for real-world deployment of general-purpose agents.