Skip to content

Agent WARPP: Workflow Adherence via Runtime Parallel Personalization

🕒 Published (v1): 2025-07-23 23:27 UTC · Source: Arxiv · Venue: ICML 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

WARPP is a training-free multi-agent framework for task-oriented dialogue (TOD) that improves LLM workflow adherence by pruning conditional branches at runtime based on user attributes and parallelizing a dedicated Personalizer agent alongside authentication. It outperforms both a no-personalization baseline and a ReAct single-agent baseline, with gains that grow as workflow complexity increases, while simultaneously reducing token usage.

Problem

LLMs executing long, conditional, multi-step workflows in TOD systems suffer from hallucinated tool calls, misordered actions, and omitted steps—especially as context length grows. Existing approaches (CoT, ReAct, multi-agent architectures) treat the workflow structure as static and do not adapt execution paths to user-specific attributes at runtime, forcing models to reason over the full combinatorial space \(b^{T/t}\) of possible execution paths, where \(b\) is the branching factor, \(T\) is total workflow token count, and \(t\) is average tokens per decision point.

Method

WARPP orchestrates four agents using OpenAI's Agents SDK:

  1. Orchestrator Agent — identifies user intent and dispatches the full workflow + tool set; launches the Personalizer and Authenticator in parallel.
  2. Authenticator Agent — runs MFA (send_verification_text / code_verifier) concurrently with personalization, masking latency.
  3. Personalizer Agent (parallelized) — applies a three-stage transformation to the full workflow using pre-fetched user attributes: (i) Static Pruning: remove branches incompatible with user attributes, inline resolved values; (ii) Fidelity Preservation: retain all outcome branches (success/failure/yes/no) around each kept tool call; (iii) Cleanup: merge descriptive steps and renumber. Returns a trimmed workflow \(W_u^*\) and a filtered tool set \(D_u^* \subseteq D\). Total complexity is \(O(T + m)\).
  4. Fulfillment Agent — receives only \(W_u^*\) and \(D_u^*\), never the full workflow; executes tool calls against simulated or real APIs.

Synthetic datasets (50 user profiles × 5 intents across Banking, Flights, Hospital) are generated via LLM + manual review; LLM-simulated clients (GPT-4o) drive interactions, and LLM judges score personalized workflow quality on relevance and completeness (1–5).

Key Contributions

  • Training-free runtime workflow pruning that reduces the effective search space from exponential to \(O(T + m)\) without fine-tuning.
  • Parallelized Personalizer agent that runs concurrently with MFA, hiding personalization latency.
  • Domain-general synthetic evaluation suite covering five intents of varying complexity (Banking: ≤5 tools/minimal branching; Flights: ≤10 tools/moderate; Hospital: >15 tools/deep branching).
  • Empirical evidence that personalization gains scale with workflow complexity—WARPP's advantage over baselines is largest on the most complex intent (Process Payment).

Results

All results are averages across GPT-4o, Llama 3, and Claude Sonnet 3.5 on 50 users per intent; bold = best per model-intent pair.

  • Update Address (simple): Exact Match — ReAct 0.73 → No Per. 0.89 → WARPP 0.97; Param Match — WARPP 98.04% vs. No Per. 99.12% (marginal regression).
  • Withdraw Retirement Funds (simple): Exact Match — WARPP 0.91; Param Match — WARPP 98.14% vs. ReAct 96.13%.
  • Book Flight (moderate): Exact Match — WARPP 0.96; Tool F1 — WARPP 99.47% vs. ReAct 96.30%.
  • Cancel Flight (moderate): No Per. outperforms WARPP on most metrics (Exact Match 0.94 vs. 0.86); WARPP still beats ReAct (0.59).
  • Process Payment (complex, Hospital): Exact Match — WARPP 0.56 vs. No Per. 0.16 and ReAct 0.16; Tool F1 — WARPP 95.46% vs. No Per. 93.52%; Param Match — WARPP 92.04% vs. No Per. 86.66%—the largest absolute gain across all intents.
  • WARPP also reduces average token usage relative to both baselines (exact figures truncated in provided text).

Limitations

  • Cancel Flight intent shows WARPP underperforming the no-personalization baseline, suggesting pruning can occasionally remove necessary branches.
  • Evaluation relies entirely on synthetic data and LLM-simulated users (GPT-4o as client); real-user studies are absent.
  • Existing TOD datasets (MultiWOZ, SGD, STAR, ABCD, SMCalFlow) were deemed insufficient, so results are not comparable to prior published benchmarks.
  • The Personalizer must complete before fulfillment begins; under elevated load, residual personalization steps bleed into the fulfillment transition, which the paper acknowledges but does not quantify.
  • Only five intents across three domains; generalization to wider enterprise workflow catalogs is unvalidated.

Relevance to Harnesses / Meta-Harnesses

WARPP is a concrete instantiation of a runtime meta-harness: it wraps a domain-specific fulfillment agent with an orchestration layer that dynamically rewrites the sub-agent's instructions and tool palette before execution begins—precisely the function a meta-harness performs. The Personalizer's three-stage workflow transformation (prune → preserve → reformat) is analogous to a harness pre-processing step that narrows the action space for downstream agents, a pattern directly applicable to harness design for complex pipelines. The parallel Authenticator + Personalizer pattern demonstrates how meta-harnesses can hide preprocessing latency by overlapping infrastructure tasks with compute-bound personalization. The \(O(T + m)\) pruning complexity result gives practitioners a concrete bound on the overhead of adding such a meta-layer.