Skip to content

Toward Reliable Design of LLM-Enabled Agentic Workflows: Optimizing Latency-Reliability-Cost Tradeoffs

🕒 Published (v1): 2026-04-21 23:09 UTC · Source: Arxiv · link

Why this paper was selected

Analyzes latency-reliability-cost tradeoffs in LLM-enabled agentic workflow design

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper develops a mathematical framework for optimizing LLM-enabled agentic workflows under simultaneous latency and cost constraints. The core result is a water-filling token allocation policy that distributes output tokens across LLM agents to maximize workflow reliability. The shadow price of the token budget has a clean economic interpretation: it equalizes marginal log-reliability gain per token across all active agents.

Problem

Agentic workflow design is currently guided by heuristics or trial-and-error, lacking principled methods to configure multi-agent pipelines (mixing LLM and non-LLM components) for optimal reliability under bounded latency and token cost. No prior formal framework captures the joint dependency of reliability, latency, and cost on per-agent token allocation.

Method

The paper models each LLM agent \(j\) with two resource variables: reasoning tokens \(X_j\) and output tokens \(L_j\). Latency is decomposed as \(\bar{T}_j = \bar{\tau}_j + X_j/\lambda_{\text{th}} + L_j/\lambda_{\text{gen}}}\). Reliability follows a parametric exponential form

\[\rho_j(X_j, L_j) = \left(1 - e^{-\alpha_j X_j}\right)\left(1 - e^{-\beta_j L_j}\right)\]

capturing diminishing marginal returns in both reasoning and output effort. Non-LLM agents have fixed \((T_i, \rho_i)\) pairs.

For a sequential workflow, overall reliability is \(R_{\text{wf}} = \prod_i \rho_i \cdot \prod_j \rho_j(X_j, L_j)\) and total latency is the sum of individual means. With \(X_j\) fixed, maximizing \(R_{\text{wf}}\) under latency budget \(B_T\) and cost budget \(C\) reduces to maximizing \(\sum_j \log(1 - e^{-\beta_j L_j})\) subject to \(\sum_j L_j/\lambda_{\text{gen}} \leq B_T\) and \(\sum_j c_{\text{tok}} L_j \leq C\). The effective budget collapses to \(B = \min\{\lambda_{\text{gen}} B_T,\ C/c_{\text{tok}}\}\), and KKT conditions yield a closed-form water-filling solution.

Key Contributions

  • Parametric performance models for LLM agents (latency, reliability, cost) as functions of reasoning and output tokens, and compact \((T_i, \rho_i)\) models for non-LLM agents.
  • Aggregation rules for sequential, parallel (conjunctive and redundant), and feedback loop workflows.
  • Proposition 1: water-filling optimal output token allocation \(L_j^* = \frac{1}{\beta_j}\log\!\left(1 + \frac{\beta_j}{\theta}\right)^+\), where \(\theta\) is the shadow price of the effective budget.
  • Corollary 1: closed-form optimal workflow reliability \(R_{\text{wf}}^* = \prod_{j} \left(1-e^{-\alpha_j X_j}\right)\frac{\beta_j}{\beta_j + \theta}\).
  • Shadow price interpretation: optimal allocation equalizes \(\frac{\partial}{\partial L_j}\log \rho_j = \theta\) across all active agents.

Results

  • Numerical experiment: \(n=5\) LLM agents, \(\beta_j \in \{0.001, 0.002, 0.0005, 0.003, 0.0015\}\), total budget \(B = 12{,}000\) tokens.
  • Water-filling allocates more tokens to agents with lower \(\beta_j\) (those that reach saturation more slowly).
  • Water-filling strictly dominates three baselines—uniform, proportional (\(\propto \beta_j\)), and inverse-proportional (\(\propto 1/\beta_j\))—across all tested token budgets (Figure 2).
  • No external benchmark dataset used; evaluation is analytical/numerical.

Limitations

  • Reasoning tokens \(X_j\) are treated as fixed exogenous parameters; joint optimization of \((X_j, L_j)\) is deferred to future work.
  • Reliability model parameters \((\alpha_j, \beta_j)\) require empirical calibration per agent; no method for estimating them is provided.
  • Analysis focuses exclusively on sequential workflows; while parallel/feedback structures are defined, their optimization is not solved.
  • The exponential reliability form is an assumption; its accuracy for real LLMs at varying token counts is not empirically validated.
  • Token budget and latency budget are treated as hard scalar constraints with no stochastic or adaptive component.

Relevance to Harnesses / Meta-Harnesses

This paper provides the first principled mathematical foundation for configuring multi-agent pipelines—exactly the class of systems that harnesses and meta-harnesses orchestrate. The water-filling allocation policy is directly actionable for a harness scheduler deciding how many output tokens to budget per sub-agent call under a global cost or latency cap. The workflow aggregation rules (sequential, parallel, feedback) formalize the composition patterns meta-harnesses already implement ad hoc, and the shadow price framework offers a concrete mechanism for dynamic resource reallocation across pipeline stages. Future extensions toward adaptive allocation (mentioned as future work) would map directly onto harness-level retry and token-budget management logic.