Skip to content

Multi-View Encoders for Performance Prediction in LLM-Based Agentic Workflows

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Sung Ju Hwang; performance prediction for LLM-based agentic workflow selection

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Agentic Predictor is a lightweight, multi-view encoder framework that predicts the performance of LLM-based agentic workflows without executing them, replacing costly LLM API calls during automated workflow search. It encodes workflows from three complementary views—graph topology, code, and prompts—and optionally uses cross-domain unsupervised pretraining (Agentic Predictor+) to compensate for scarce labeled data. Averaged across three domains it improves prediction accuracy by up to 6.90% and workflow utility by up to 5.87% over graph-based baselines.

Problem

Automated agentic workflow design requires evaluating large numbers of candidate configurations, each demanding repeated LLM API calls. This creates two compounding bottlenecks: (1) workflow heterogeneity—subtle variations in agent topology, prompting strategy, and tool usage produce dramatically different behaviors, making a unified predictive model hard to learn; and (2) label scarcity—generating ground-truth success/failure labels via execution is prohibitively expensive, leaving supervised approaches data-starved.

Method

A workflow is formalized as \(W = \{V, E, P, C\}\): a DAG of \(N\) agents with edges \(E\), per-agent system prompts \(P\), and full workflow code \(C\). The goal is a predictor \(\hat{e} = M_\Theta(W, T)\) that estimates performance on task description \(T\) without executing the workflow.

Multi-view encoding produces three complementary representations: - Graph encoder: constructs three GNN-processed views (\(G_\text{prompt}\), \(G_\text{code}\), \(G_\text{operator}\)) sharing the same DAG topology but with different node features. Cross-view self-attention \(\hat{X} = \text{LN}(\text{MHA}(X,X,X)+X)\) is applied across the view axis per node, followed by view-attention pooling \(H = \text{ViewAttnPool}(\hat{X})\) and graph readout to yield \(Z_G\). - Code encoder: an \(L\)-layer MLP over the entire workflow code \(C\), yielding \(Z_C = \text{MLP}_C(C)\). - Prompt encoder: an \(L\)-layer MLP over all concatenated system prompts, yielding \(Z_P = \text{MLP}_P(P)\). - Aggregation: \(Z = \text{MLP}([Z_G; Z_C; Z_P])\).

Cross-domain unsupervised pretraining (Agentic Predictor+) trains the encoder on \(M\) unlabeled workflows with a combined objective \(\mathcal{L}_\text{enc} = \mathcal{L}_\text{rec} + \mathcal{L}_\text{con}\), where \(\mathcal{L}_\text{rec}\) is a reconstruction loss over graph, code, and prompt modalities, and \(\mathcal{L}_\text{con}\) is a cross-modal contrastive loss treating same-workflow embeddings from different view pairs \((G,C)\), \((G,P)\), \((C,P)\) as positives and other in-batch configurations as negatives.

Performance predictor: a lightweight MLP head \(M_\Theta\) trained on a small labeled set using the joint representation \(F = [Z; T]\), where \(T\) is a task embedding from a pretrained encoder (T5 or BERT). At search time, \(K\) candidate workflows are scored by the predictor; only the top-\(k\) are executed, dramatically reducing LLM call volume.

Key Contributions

  • Multi-view workflow encoding combining graph (agent topology + tool-call patterns), code (program semantics), and prompt (role/behavioral semantics) views with cross-view attention fusion.
  • Cross-domain unsupervised pretraining with joint reconstruction and cross-modal contrastive objectives to enable low-data-regime predictor training.
  • Agentic Predictor framework that is search-agnostic and functions as a drop-in ranker for any automated workflow search algorithm.
  • A curated three-domain benchmark for evaluating performance predictors on heterogeneous agentic workflows.

Results

  • Averaged across three domains, Agentic Predictor improves prediction accuracy by up to 6.90% over strong graph-based baselines.
  • Improves workflow utility (ranking quality) by up to 5.87% over the same baselines.
  • Outperforms FLORA-Bench-style single-view GNN predictors and other graph-based baselines in both metrics.
  • (Full per-domain tables are in the truncated portion of the paper and not reproduced here.)

Limitations

  • Paper text is truncated; complete ablation and per-domain numbers are not available for this digest.
  • Cross-domain unsupervised pretraining requires access to a pool of unlabeled workflows from related domains, which may not always be available for highly specialized or novel domains.
  • The code and prompt encoders use flat MLPs over pooled embeddings; long or complex code/prompt structures may lose fine-grained information.
  • The search component is validated only with random search as the base optimizer; integration with more structured search algorithms (evolutionary, Bayesian) is deferred to appendix results.
  • The predictor frames evaluation as classification (pass/fail) or regression over aggregate success rate, which may obscure partial-success or latency tradeoffs.

Relevance to Agentic AI / LLM Agents

Automated agentic workflow design is an active and resource-intensive frontier: frameworks like ADAS, AFlow, and AgentSquare each require many LLM calls per candidate. Agentic Predictor directly attacks this bottleneck by decoupling performance estimation from execution, analogously to how NAS performance predictors transformed neural architecture search. The multi-view encoding principle—capturing topology, code logic, and prompt semantics jointly—is a transferable design pattern for any system that needs to represent heterogeneous agent configurations as a structured artifact. For researchers building or studying automated multi-agent systems, this work provides both a benchmark and a practical recipe for cheap, label-efficient workflow selection.