Multi-Agent LLMs for Adaptive Acquisition in Bayesian Optimization¶

🕒 Published (v1): 2026-03-30 20:05 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Single-agent LLMs suffer from "cognitive overload" when jointly selecting an exploration–exploitation strategy and generating candidates in black-box optimization. This paper decomposes the task into a two-agent harness: a strategy agent that assigns interpretable weights over search criteria (\(w_t\) over exploitation, informativeness, diversity, representativeness) and a generation agent that produces candidates conditioned on those weights. The decomposition stabilizes search dynamics and improves performance on weakly structured benchmarks while revealing systematic failures on smooth landscapes.

Problem¶

LLM-based optimizers treat exploration–exploitation as implicit, prompt-encoded reasoning, making search behavior opaque and uncontrollable. Single-agent designs that jointly perform policy selection and candidate generation conflate abstract strategic reasoning with precise numerical execution, producing unstable trajectories and premature convergence—pathologies that existing hybrid methods (LLAMBO, BOLT, BOPRO, OPRO) do not resolve because strategy remains entangled with generation.

Method¶

At each iteration \(t\), the framework runs a two-stage loop: 1. Strategy agent receives the optimization history \(D_{t-1}\), metric definitions (exploitation, informativeness, diversity, representativeness), and behavioral summaries; outputs a normalized weight vector \(W_t \in \mathbb{R}^4\) via structured prompting. 2. Generation agent receives \(D_{t-1}\) and \(W_t\); produces candidate \(x_t\) by optimizing a scalarized score derived from the weighted criteria.

Metrics are computed externally from the optimization history and candidate set (e.g., local sparsity for informativeness, dispersion for diversity, proximity to cluster centers for representativeness) and handed to the LLM as inputs—no parameter updates occur. LangChain is used for structured prompt templating with fixed output schemas to ensure machine-readable outputs and reproducibility. Experiments use llama-3.3-70b-instruct on three benchmarks (Rosenbrock, hyperparameter tuning, robot pushing) with a 30-query budget.

Key Contributions¶

A metric-level framework for analyzing exploration–exploitation behavior in LLM-driven BO under three operationalizations of exploration: informativeness, diversity, and representativeness.
A two-agent decomposition that separates strategic policy mediation from tactical candidate generation, rendering the trade-off explicit and observable at every iteration via interpretable weight trajectories.
Empirical characterization of when LLMs succeed or fail as search policy mediators: competitive with EI/UCB on irregular/discontinuous landscapes, systematically inferior on smooth analytical functions.

Results¶

Rosenbrock (smooth): EI and UCB converge cleanly; multi-agent LLM is more stable than single-agent but still fails to achieve fine-grained numerical exploitation; strategy weights misinterpret slow progress as stagnation and over-shift to exploration.
Hyperparameter tuning (discrete, high-dim): Multi-agent LLM achieves competitive or superior performance to EI/UCB in some runs; single-agent remains unstable; weights show persistent exploitation emphasis.
Robot pushing (discontinuous control): Multi-agent LLM outperforms single-agent and approaches EI/UCB; early exploration followed by exploitation-dominant policy matches landscape demands.
Paired-metric ablations (Rosenbrock): Restricting the strategy agent to one auxiliary criterion improves performance; strategy weights adaptively reallocate toward exploration during stagnation.
Paired-metric ablations (robot pushing): All single-pair configurations fail to improve over single-agent; weight trajectories show premature stagnation, indicating multiple criteria are needed for complex landscapes.

Limitations¶

Experiments capped at 30 function queries; behavior under larger budgets is unknown.
All results use a single model (llama-3.3-70b-instruct); generalizability across model families is untested.
The metric set (four criteria) is fixed by design; richer or adaptive metric discovery is left to future work.
LLM-mediated strategies fail to recover classical acquisition function behavior (e.g., EI's fine-grained exploitation), particularly on smooth, low-noise landscapes.
No theoretical regret guarantees are provided; the framework is purely empirical.
Prompt sensitivity analysis is qualitative; quantitative robustness to prompt variations is not measured.

Relevance to Harnesses / Meta-Harnesses¶

This paper is a direct instantiation of an LLM meta-harness pattern: rather than calling a single monolithic prompt, it orchestrates a fixed two-stage pipeline (strategy agent → generation agent) where the first agent produces structured intermediate state (\(W_t\)) consumed by the second. The explicit weight vector acts as the inter-agent communication contract, making the harness's internal state inspectable and adjustable—a key property for harness designers. The finding that cognitive overload degrades single-agent performance while decomposition recovers stability directly motivates role specialization in multi-agent harness architectures. The structured prompt templating via LangChain and fixed output schemas demonstrate concrete harness engineering choices (serialization format, context management, output parsing) that are broadly applicable beyond BO.