Skip to content

AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search

🕒 Published (v1): 2025-06-06 12:07 UTC · Source: Arxiv · Venue: AAAI · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgentSwift is an automated agent design framework that formalizes a hierarchical search space jointly covering agentic workflows and composable functional components (memory, tool use, planning), trains a lightweight 7B value model to cheaply predict agent performance, and navigates the space with uncertainty-guided Monte Carlo Tree Search (MCTS). Evaluated on seven benchmarks, it achieves an average 8.34% gain over prior automated agent search baselines while requiring far fewer real evaluations.

Problem

Existing automated agent design methods (AFlow, ADAS, AgentSquare) suffer from three compounding deficiencies: (1) search spaces restricted to workflow topology, leaving human-designed functional components (memory, planning, tool use) unoptimized; (2) prohibitively expensive evaluations — a single CoT agent on ALFWorld costs ~$60 with GPT-4o, forcing methods to evaluate every candidate fully; and (3) inefficient exploration of the resulting large combinatorial design space.

Method

AgentSwift defines an agent as \(A = (W, M, T, P)\) where \(W\) is an agentic workflow (directed graph of LLM-invoking nodes), and \(M, T, P\) are composable memory, tool-use, and planning components, each with their own prompt/temperature/backend parameters. The joint search space \(S_{\text{agent}}\) subsumes AFlow and AgentSquare as special cases.

Value model. A 7B language model (\(f_\theta\)) is fine-tuned to predict \(\hat{v} = f_\theta(A, d)\) (performance on task \(d\)). Training data (220 samples) is constructed in two stages: a \(t{=}2\) pairwise covering array ensures all pairwise \((W, M, T, P)\) interactions appear at least once; then Balanced-Extreme Bayesian Optimization fits a Gaussian Process with Hamming kernel and samples via both UCB \((\mu + \kappa\sigma)\) and LCB \((-\mu + \kappa\sigma)\) acquisition functions to populate high- and low-performing regions for discriminative diversity.

Uncertainty-guided MCTS. Selection uses a soft mixed probability blending observed performance \(s_i\) and epistemic uncertainty \(u_i = |s_{\text{real}} - \hat{s}|\): $\(P_{\text{mixed}}(i) = \lambda \cdot \frac{1}{n} + (1{-}\lambda) \cdot \frac{\exp(\alpha((1{-}\beta)s_i + \beta u_i - s_{\max}))}{\sum_j \exp(\alpha((1{-}\beta)s_j + \beta u_j - s_{\max}))}\)$

Expansion applies three hierarchical operations — recombination (swap one subsystem from pool), mutation (LLM generates a novel subsystem implementation), and refinement (targeted edits informed by failure cases) — at both the workflow and component levels. The value model ranks candidates within each step to advance only the most promising design.

Key Contributions

  • Hierarchical search space \(S_{\text{agent}}\) jointly optimizing workflow structure and three composable functional components, generalizing prior workflow-only methods.
  • Dataset construction combining \(t{=}2\) covering arrays with Balanced-Extreme Bayesian Optimization (UCB+LCB) to yield a 220-sample training set with broad coverage and discriminative diversity.
  • Uncertainty-guided MCTS with three-step hierarchical expansion (recombination → mutation → refinement) using the value model as an in-loop surrogate evaluator.
  • Empirical validation across 7 benchmarks (ALFWorld, SciWorld, MATH, WebShop, M3Tool, Travel, PDDL) spanning embodied, math, web, tool, and game domains.

Results

All results use GPT-4o-mini; evaluation budget capped at 60 agents per method; averaged over 3 runs.

  • AgentSwift scores: ALFWorld 0.806, SciWorld 0.509, MATH 0.628, WebShop 0.562, M3Tool 0.634, Travel 0.573, PDDL 0.614.
  • Best prior automated method (AgentSquare): ALFWorld 0.701, SciWorld 0.475, MATH 0.556, WebShop 0.520, M3Tool 0.561, Travel 0.553, PDDL 0.577.
  • Average improvement of 8.34% over both automated search baselines (AgentSquare, AFlow, ADAS, MaAS) and manually designed agents (CoT, ToT, Voyager, DEPS, etc.).
  • Steeper and more stable search trajectories than all baselines; high-performing agents found with significantly fewer real evaluations.
  • Value model generalizes to unseen tasks with minimal fine-tuning; discovered agents transfer across GPT-4o, GPT-4o-mini, and DeepSeek-v3 backbones.

Limitations

  • Value model training still requires 220 real evaluations upfront; this cost is amortized but not eliminated.
  • Evaluation budget fixed at 60 agents for fair comparison — behavior at larger budgets is not characterized.
  • Functional component vocabulary is closed (memory, tool use, planning); extending to new component types requires new pool construction and retraining.
  • Covering array and Bayesian sampling strategy are designed for the fixed four-element agent tuple; generalization to richer hierarchies is not demonstrated.
  • Uncertainty is measured post-hoc as \(|s_{\text{real}} - \hat{s}|\), so every MCTS leaf still requires one real evaluation, limiting savings for very expensive benchmarks.

Relevance to Harnesses / Meta-Harnesses

AgentSwift is architecturally a meta-harness: it orchestrates a pipeline of LLM sub-agents (proposer \(\pi_\theta\), programmer \(\pi_\xi\), refiner \(\pi_\phi\)) and a learned value model to generate, score, and iteratively improve child agent harnesses. The MCTS loop over a hierarchical design space directly instantiates the meta-harness pattern of a controller that discovers and composes sub-harnesses rather than hard-coding them. The value model as surrogate evaluator is a reusable architectural primitive for any meta-harness that must cheaply rank a large candidate pool — analogous to predictor-based NAS in the neural architecture search literature. For researchers tracking harness automation, AgentSwift provides a concrete, evaluated template: formalize the search space, train a cheap ranker on covering-array samples, and drive search with uncertainty-aware MCTS.