Flow-of-Options: Diversified and Improved LLM Reasoning by Thinking Through Options

🕒 Published (v1): 2025-02-18 15:11 UTC · Source: Arxiv · Venue: ICML 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Flow-of-Options (FoO) replaces ad-hoc LLM chain/tree reasoning with a directed acyclic graph (DAG) whose nodes are enumerated solution options per task step, forcing systematic diversity in LLM-driven agentic ML workflows. Edge values are updated via max propagation after executing each sampled "walk," and beam traversal selects top-valued option combinations. Combined with Case-Based Reasoning (CBR), the framework achieves 38.2–69.2% rank improvement on data science tasks and 37.4–47.9% on therapeutic chemistry tasks over state-of-the-art baselines, at under $1 per task.

Problem

LLMs exhibit strong pre-training biases that persist under standard prompting and even chain-of-thought: e.g., GPT-4o and o1 default to RandomForest in nearly all Sklearn-based ML tasks regardless of the problem. Existing agentic frameworks (AutoGPT, LangChain) fail to overcome these biases; tree-based search (SELA/MCTS) is computationally intensive and cannot share credit across branches; DAG-based approaches (Data Interpreter) do not enforce diversity or guarantee acyclicity. DS-Agent works around the bias via a Kaggle-curated human insight repository, which is unavailable for most domains.

Method

FoO Construction. Given a task \(T = \{s_1, \ldots, s_n\}\), \(k\) options are sampled per step \(i\) from the LLM, conditioned on the task and previously generated options: \(o_k^{(i)} \sim p_\theta(T, s_i, o_k^{(1:i-1)})\). The resulting structure is a fully connected DAG \(F = (V, E, r)\) of depth \(n\) where every node at depth \(i-1\) connects to every node at depth \(i\); edges are initialized with a small fixed value.

Traversal and Value Update. A walk \(W = (o^{(1)}, \ldots, o^{(n)})\) samples one option per depth. After executing walk \(W_j\) with outcome \(R_j = f(W_j)\), edge values are updated by max propagation: $\(r(o_k^{(i)}, o_k^{(i+1)}) = \max(r(o_k^{(i)}, o_k^{(i+1)}),\; R)\)$ Node value is defined as \(\text{value}(v) = \max_{u \in V^{(i-1)}} r(u,v)\).

Beam Traversal. Walks are generated by sampling from the top-\(b\) valued nodes at each depth (beam width \(b\)); \(b\) is reduced over \(T\) iterations to shift from exploration to exploitation.

Consistency Checking. An LLM consistency checker filters walks with contradictory option combinations (e.g., model choice at depth \(i\) conflicts with downstream step) before execution.

Agentic Framework with CBR. A Planner LLM generates and filters plan steps to \(n\) impactful ones. In development, \(j\) walks are executed in parallel per iteration; the resultant FoO is stored as case \(c = (T, F, R^*)\). In deployment, the most similar stored FoO (by cosine similarity of task embeddings) is retrieved, adapted via an Adapter LLM, and directly executed with \(k=0\), \(b=1\), \(T=1\), eliminating new option generation. Pruning removes low-value options to bound DAG growth.

Key Contributions

• FoO data structure: fully connected DAG with max-update propagation that explicitly enumerates and credits option combinations across plan steps.
• Diversity enforcement: option generation is conditioned to be distinct from prior options, with empirical validation that output distribution broadens vs. zero-shot.
• Development/Deployment split with CBR: compressed FoO networks serve as transferable, domain-agnostic task summaries; deployment cost is ~$0.03/task.
• LLM-free graph construction: cycles are structurally impossible since LLMs only generate node content, not graph topology.
• Broad generalization: system demonstrated on tabular classification/regression, ADME-Tox chemistry, reinforcement learning (cartpole), image generation (MNIST), clustering, machine translation, and TSP.

Results

• DS tasks (16 benchmarks): average rank 1.44 vs. Data Interpreter 2.33, Autogen/zero-shot 3.19, SELA 4.17, DS-Agent 3.69, AutoGluon 4.67 — rank improvement of 38.2% over DS-Agent to 69.2% over AutoGluon.
• TDC ADME-Tox (17 tasks): average rank 1.47 vs. DeepMol 2.35, Autogen 2.71, zero-shot 2.82 — rank improvement of 37.4–47.9%.
• Cost: development avg $0.62, deployment avg $0.03; both well under $1/task using GPT-4o.
• Time: development 13.29 min avg, deployment 0.91 min avg (SELA: 21 min, DS-Agent development: 12 min).
• 100% task success in all methods except DI (which failed the W task entirely); FoO fails only on AR task.
• Scalability: extended to drug-drug combination and chemical bond prediction with ~100× larger datasets than prior tabular tasks.

Limitations

• Naive walk sampling can produce duplicate walks across iterations; authors acknowledge this and propose informed sampling as future work.
• Option pruning (removing 2 low-value options per depth per iteration) may discard options that would be valuable in combination with later steps.
• All experiments use GPT-4o as the sole backbone LLM; generalization to other LLMs is not empirically evaluated.
• The beam-width schedule and hyperparameters (\(k\), \(j\), \(T\), \(n\)) require per-domain tuning.
• CBR retrieval relies on embedding similarity with a fixed threshold; failure cases where the closest case is mismatched (but above threshold) are not deeply analyzed.

Relevance to Harnesses / Meta-Harnesses

FoO is directly a meta-harness pattern: it orchestrates five sub-agents (Planner, Option Generator, Plan Executor, Consistency Checker, Adapter) under a structured DAG that acts as the meta-level control mechanism determining which options to explore and how to propagate performance signal across runs. The development/deployment split with a persistent case bank implements a harness-level memory layer — the FoO network itself becomes a compressed, reusable task representation transferable across runs and tasks. The max-update value propagation is analogous to harness-level credit assignment, decoupling individual option quality from combination quality — a problem directly relevant to multi-agent orchestration in meta-harnesses. For practitioners building harnesses, FoO provides a concrete, cost-tractable template for diversity enforcement and iterative refinement within an agent loop.