Skip to content

When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems

🕒 Published (v1): 2026-05-28 15:45 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML; design principles and lessons from hybrid cloud-device agent systems

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper systematically studies hybrid multi-agent systems (MASs) that pair cloud-hosted frontier LLMs with on-device small language models (SLMs) as executor agents. Two architectures—PEVR (Plan–Execute–Verify–Replan) and EVA (Execute–Verify–Advise)—are evaluated on deep search and UI assistance benchmarks across accuracy, cloud API cost, and edge energy consumption. Key finding: no single architecture dominates across tasks, and more cloud compute does not monotonically improve performance.

Problem

Agentic LLM workloads are long-horizon and token-intensive, making full reliance on frontier cloud models expensive, while SLMs alone have performance and context-length gaps that limit long-horizon reliability. Hybrid cloud–edge MAS architectures offer a natural middle ground but have been designed ad hoc without principled study of how individual design choices (model assignment, supervision frequency, restart strategy) shift the Pareto frontier of accuracy, cost, and energy.

Method

Two representative hybrid MAS architectures are implemented and studied:

PEVR (Plan–Execute–Verify–Replan): A cloud Supervisor generates an initial plan \(p_0\); an on-device Executor runs a ReAct loop. Every \(T_v\) steps, the Supervisor verifies plan adherence and, upon detecting misalignment, issues a revised plan \(p_t^{\text{new}}\), which replaces the Executor's current plan and resets its context.

EVA (Execute–Verify–Advise): The Executor starts without a plan. Every \(T_v\) steps the cloud Supervisor checks progress and, when needed, provides (i) a compressed summary of past actions and (ii) advisory feedback \(h_t^{\text{new}}\); the Executor's context is reset to this summary.

In both architectures, execution (token-heavy) stays on-device; cloud involvement is sparse and controlled by the verification interval \(T_v\). Context resets on intervention prevent unbounded KV-cache growth. Experiments use GPT-4o as cloud agent and Qwen3-4B/8B/14B/32B as edge executors; benchmarks are HotpotQA, FanOutQA (deep search), and AppWorld (stateful UI assistance). Efficiency is measured in API cost (USD) and estimated edge energy (Joules).

Key Contributions

  • Unified cloud–edge MAS framework with two representative architectures (PEVR, EVA) and a systematic sweep over model size and verification interval \(T_v\).
  • Empirical demonstration that hybrid MASs reliably Pareto-dominate both monolithic edge and monolithic cloud baselines across all three benchmarks.
  • Finding that architecture superiority is task-dependent: PEVR dominates on stateful UI tasks (AppWorld); EVA dominates on open-ended deep search (HotpotQA, FanOutQA).
  • Mechanistic analysis showing that PEVR's plan-based verifier has higher false positives (over-intervenes), which is benign in UI tasks but catastrophic in deep search where restarts degrade accuracy.
  • Context efficiency result: intervention-triggered resets reduce peak KV-cache footprint by up to \(3\times\) compared to monolithic edge agents, with EVA yielding larger energy savings due to summarization.
  • Ablation showing that assigning the cloud model as executor (symmetric swap) is strictly worse in both accuracy and cost, validating the design principle of confining execution to the edge.

Results

  • Hybrid vs. monolithic: For all benchmarks and edge model sizes, at least one MAS configuration (PEVR or EVA) Pareto-dominates the monolithic edge baseline at lower cost than the monolithic cloud baseline.
  • AppWorld (UI, stateful): PEVR substantially outperforms EVA; e.g., with Qwen3-32B executor, PEVR Task Success = 0.25 vs. EVA lower; monolithic cloud (GPT-4o) = 0.37 at $0.14 cost, PEVR with Qwen3-32B reaches 0.25 at $0.09.
  • FanOutQA (deep search): EVA substantially outperforms PEVR; PEVR often underperforms even the monolithic edge baseline when \(T_v\) is small. EVA ROUGE-1 F1 comparable to or exceeding monolithic edge at a fraction of cloud cost.
  • Energy reduction: Context resets reduce edge energy consumption by up to \(3\times\) vs. monolithic edge; EVA typically achieves larger savings than PEVR.
  • Verifier false positives (Qwen3-32B, \(T_v=3\)): PEVR false positive rate = 6.2% (AppWorld), 7.7% (FanOutQA); EVA false positive rate = 1.9% (AppWorld), 6.1% (FanOutQA).
  • Symmetric swap (cloud executor, edge supervisor): Task Success on AppWorld drops (e.g., GPT-4o executor + Qwen32B supervisor = 0.25 Task Succ. but $0.67 cost, vs. PEVR at 0.25 for $0.09); confirms edge-executor design is cost-optimal.
  • Executor size correlates positively with performance across all configurations.

Limitations

  • Edge energy consumption is estimated via a simple analytic model rather than real on-device deployment and measurement.
  • Only one cloud backbone (GPT-4o) is evaluated; results may not generalize to other frontier models.
  • Only two MAS architectures are studied; the design space (e.g., peer-to-peer, decentralized topologies) is much larger.
  • Optimal verification interval \(T_v\) is task-dependent with no automatic selection mechanism; in practice a user must tune it.
  • FanOutQA and HotpotQA performance is measured by ROUGE-1 F1, which may not fully capture answer correctness for complex multi-hop queries.
  • The paper focuses on English-language, API-accessible benchmarks; generalization to low-resource, multimodal, or real on-device deployments is unexamined.

Relevance to Agentic AI / LLM Agents

This work directly addresses the practical deployment constraint faced by long-horizon LLM agents: frontier cloud models are powerful but costly, while capable SLMs are constrained by context length and DRAM. The systematic comparison of PEVR and EVA architectures provides the agentic AI community with actionable design principles—specifically, that plan-based orchestration suits stateful, recoverable-from-plan tasks, while advisory summarization suits open-ended retrieval tasks where over-intervention is harmful. The finding that greater cloud compute does not monotonically improve performance challenges a common assumption in multi-agent system design and motivates adaptive supervision policies. The context-reset mechanism is broadly applicable to any long-horizon agent that must run under memory constraints, making this a useful reference for edge-deployed agentic pipelines.