Multi-View Encoders for Performance Prediction in LLM-Based Agentic Workflows¶

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR KAIST; multi-view performance prediction for agentic workflow configuration optimization

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Agentic Predictor is a lightweight, multi-view encoder framework that predicts the performance of LLM-based agentic workflows without executing them, replacing costly trial-and-error LLM API calls with a learned surrogate. It encodes workflows simultaneously from graph topology, code structure, and system-prompt semantics, and uses cross-domain unsupervised pretraining to overcome labeled-data scarcity. On a three-domain benchmark it outperforms strong graph-based baselines in both accuracy and workflow utility.

Problem¶

Automated design of multi-agent workflows (ADAS, AFlow, AgentSquare, etc.) requires repeatedly invoking LLM APIs to evaluate each candidate configuration, making the search prohibitively expensive. Two compounding factors are: (1) workflow heterogeneity—subtle differences in agent topology, prompts, and tool usage cause large performance swings that a single-view representation cannot capture; and (2) labeled-data scarcity—generating ground-truth execution outcomes (pass/fail rates) is expensive, so supervised training data is severely limited.

Method¶

The framework, Agentic Predictor, treats performance prediction as $\hat{e} = M_\Theta(W, T)$ where $W = \{V, E, P, C\}$ is the workflow DAG and $T$ is the task description.

Multi-view workflow encoding fuses three complementary encoders: - Graph Encoder: three GNN-encoded sub-graphs ($G_{\text{prompt}}$, $G_{\text{code}}$, $G_{\text{operator}}$) with shared topology; node embeddings are stacked into $X \in \mathbb{R}^{N \times 3 \times d}$, enriched via cross-view multi-head self-attention, then fused via view-attention pooling and graph readout to yield $Z_G$. - Code Encoder: an $L$-layer MLP over the full workflow code string, $Z_C = \mathrm{MLP}_C(C)$, capturing global program-level semantics and tool-usage patterns. - Prompt Encoder: an $L$-layer MLP over concatenated system/instruction prompts, $Z_P = \mathrm{MLP}_P(P)$.

The three representations are concatenated and fused: $Z = \mathrm{MLP}([Z_G, Z_C, Z_P])$.

Cross-domain unsupervised pretraining (Agentic Predictor+) trains the encoder on $M$ unlabeled workflows with a combined loss: $$\mathcal{L}_{\text{enc}} = \mathcal{L}_{\text{rec}} + \mathcal{L}_{\text{con}}$$ where $\mathcal{L}_{\text{rec}}$ is an $\ell_2$ reconstruction loss per modality and $\mathcal{L}_{\text{con}}$ is a cross-modal InfoNCE contrastive loss using index-aligned cross-view positives (e.g., graph–prompt, graph–code, code–prompt pairs) and in-batch negatives.

Performance predictor: a lightweight MLP head $M_\Theta$ trained on small labeled sets with the joint feature $F = [Z, T]$ (workflow embedding concatenated with a T5/BERT task embedding), minimizing binary cross-entropy or MSE depending on label type.

Predictor-guided search: random search samples $K$ candidates; the predictor ranks them; top-$k$ are selected for actual execution, bypassing full LLM evaluation for the rest.

Key Contributions¶

Multi-view workflow encoding jointly capturing graph topology, code structure, and prompt semantics—addressing heterogeneity where single-view GNNs fail.
Cross-domain unsupervised pretraining with reconstruction + cross-modal contrastive objectives to overcome labeled-data scarcity.
Agentic Predictor framework unifying both components as a search-agnostic, lightweight surrogate for agentic workflow evaluation.
Empirical benchmark spanning three domains demonstrating up to 6.90% accuracy gain and 5.87% utility gain over strong graph-based baselines.

Results¶

Averaged across three domains, Agentic Predictor improves prediction accuracy by up to 6.90% and workflow utility (ranking quality) by up to 5.87% over strong graph-based baselines (including FLORA-Bench's GNN predictor).
The pretrained variant (Agentic Predictor+) achieves these gains in the low-label regime, where baselines without pretraining degrade substantially.
Additional workflow optimization experiments appear in the paper's Appendix B.6 (not reproduced in the provided text).

(The paper text is truncated before the full results tables; specific per-domain numbers and all baseline names are not available in the provided excerpt.)

Limitations¶

Evaluation is confined to three domains—generalization to highly specialized or novel agentic workflow spaces is untested.
The predictor is trained to classify/rank configurations; it does not generate or repair workflows, so it must be paired with a separate search algorithm.
Labeled data, though reduced, is still required for fine-tuning; cold-start prediction on entirely unseen task types may degrade.
The framework assumes workflows are representable as DAGs; cyclic or stateful agent graphs fall outside the formalism.
Computational cost of the pretraining phase itself (over $M$ unlabeled workflows) is not quantified in the provided text.

Relevance to Harnesses / Meta-Harnesses¶

Agentic Predictor is directly relevant as a meta-harness surrogate: it wraps the inner execution harness of an agentic workflow system and replaces costly end-to-end runs with a learned approximator, enabling the meta-harness to conduct large-scale configuration search economically. The predictor-guided search loop (sample → score via predictor → select top-$k$ → execute) is itself a meta-harness pattern, analogous to NAS performance predictors sitting above the training loop. The cross-domain pretraining design also mirrors meta-harness concerns about transfer—learning generalizable priors over workflow structure so that a new task domain requires only a small labeled fine-tuning set rather than full re-evaluation. Researchers building harnesses for automated agent design (ADAS, AFlow-style) can integrate Agentic Predictor as a plug-in ranking oracle to reduce the per-iteration API budget.