Beyond Static Endpoints: Tool Programs as an Interface for Flexible Agentic Web Services

🕒 Published (v1): 2026-06-18 09:31 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; rethinks tool interfaces for long-horizon agentic web workflows

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ToolPro replaces static API endpoint sequences with executable tool programs as the agent–service interface, allowing LLM agents to submit an entire multi-step workflow as a single compiled object rather than issuing repeated round-trip calls. It achieves up to 53.4% end-to-end latency reduction and 96.1% client-side traffic reduction by consolidating multi-step interactions server-side, with exactly-once semantics for state-modifying operations.

Problem

LLM agents orchestrating web services must currently decompose long-horizon workflows (loops, conditionals, intermediate bindings) into brittle sequences of static endpoint calls. Each step requires a client–service round trip plus an LLM reasoning round, causing latency that grows linearly with workflow length, systematic over-/under-fetching (since control logic lives outside the service), and cascading partial-failure states where naïve retries duplicate state-modifying side effects.

Method

ToolPro represents a workflow as an effect-typed tool program \(P\) — a procedure whose only external actions are \(\texttt{CALL}(e, a)\) stubs annotated with \(\text{eff}(e) \in \{\text{READ}, \text{WRITE}\}\) — and submits it to a service-side runtime for execution. The system has three core mechanisms:

1. Constraint-guided construction (addresses executability): The client synthesizes \(P\) and applies lightweight structural checks (endpoint coverage, control-flow skeleton, value-flow sanity). The server projects \(P\) via a deterministic \(\Pi(\cdot)\) into a constrained surface (structured control flow only; no threads, dynamic linking, or ambient I/O), then compiles and executes in a WebAssembly sandbox. Failures trigger bounded in-place repair using compiler diagnostics and runtime traces, localized to the offending call site or minimal block.

2. Effect-aware replay (addresses retry safety): The runtime maintains an ordered history log \(H\) of completed WRITE outcomes and a per-execution working log \(W\). On re-execution, each WRITE call \((e, a)\) is matched against the earliest unused \(H\) entry; if matched, the cached outcome is returned without re-emitting the call. Formally, each dynamic WRITE is emitted to the underlying service at most once (Proposition 3.1). If a repair changes the committed WRITE prefix ordering, replay is disabled and ToolPro falls back to stepwise calling.

3. Profile-driven consolidation policy (addresses when to consolidate): ToolPro tracks moving averages of \(T_{\text{RTT}}\), \(T_{\text{DEC}}\) (per-step client decision overhead), and \(T_{\text{BUILD}}\) (program construction cost). It executes a tool program only when the predicted net benefit $\(\Delta T = (N-1)\cdot(T_{\text{RTT}} + T_{\text{DEC}}) - T_{\text{BUILD}} > 0,\)$ where \(N\) is the estimated number of dynamic endpoint invocations. Cold-start bootstraps with stepwise runs; after stabilization, per-instance mode selection is applied.

Key Contributions

• Identifies static endpoint sequences as a representational bottleneck for agentic web workflows and proposes tool programs as a first-class interface abstraction.
• ToolPro system operationalizing tool programs over MCP-style services with WebAssembly sandboxing — the first agentic web service runtime with exactly-once WRITE semantics.
• Constraint-guided construction pipeline (client synthesis + server projection + bounded in-place repair with verifiable feedback).
• Effect-aware replay algorithm that prevents duplicated WRITE side effects across repair-driven re-executions.
• Profile-driven consolidation rule that adaptively selects between program execution and stepwise calling per instance.

Results

• End-to-end latency: up to 53.4% reduction over stepwise MCP baseline.
• Client-side traffic: up to 96.1% reduction.
• Gains increase with higher network latency and longer workflow complexity (larger \(N\)).
• Evaluated on three open-source applications (Memos, Directus, MinIO) exposed as MCP-style services with read-only (.r) and read-write (.w) parameterized workflows.
• Supplemental realistic workflows stress nondeterministic retrieval, branching, coordinated writes, non-idempotent effects, and cross-service execution; task accuracy also reported there.
• ToolPro-step (no consolidation, intent-structured guidance) isolates the gain from replacing replanning with program submission alone.

Limitations

• Consolidation benefit depends on \(N\) and network latency; for short workflows or low-latency networks, construction overhead can negate savings (handled by the consolidation policy, but adds cold-start complexity).
• Effect-aware replay requires endpoint effect labels (\(\text{READ}/\text{WRITE}\)) declared at every call site; services without such annotations or with meaningful nondeterminism under identical arguments fall back to stepwise calling.
• Repair is bounded by an attempt budget; programs that fail to converge under the budget revert to stepwise calling, losing the efficiency benefit.
• The constrained surface excludes arbitrary application logic, so workflows requiring dynamic dispatch, complex data structures, or external I/O beyond the service boundary cannot be expressed.
• Evaluation is over three applications with synthetic parameterized workflows; broader generalization across diverse real-world service APIs is not fully demonstrated.

Relevance to Agentic AI / LLM Agents

ToolPro addresses a fundamental interface mismatch between how LLM agents reason (holistic plans) and how current web services expose themselves (atomic endpoints), a problem that becomes increasingly acute as agents take on longer-horizon tasks. The shift from reactive endpoint selection to program submission is architecturally significant: it moves control-flow execution server-side, reducing the reasoning burden on the agent and decoupling task intent from execution mechanics. The effect-aware replay mechanism directly tackles a correctness hazard inherent to any agent that retries tool calls — duplicate WRITE effects — which is underexplored in the tool-use literature. The MCP-style instantiation positions this work as directly applicable to the growing ecosystem of agent tool servers.