Skip to content

EcoAgent: An Efficient Device-Cloud Collaborative Multi-Agent Framework for Mobile Automation

🕒 Published (v1): 2025-05-08 17:31 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EcoAgent is a closed-loop device–cloud collaborative multi-agent framework for mobile automation that deploys a cloud-based Planning Agent alongside two lightweight on-device agents (Execution and Observation). It matches fully cloud-based success rates on AndroidWorld while cutting cloud token usage by up to 96% and reducing per-step latency to 3.9 s. Privacy is preserved by converting screenshots to compact text summaries before any cloud transmission.

Problem

Existing mobile multi-agent systems are either fully cloud-based (high latency, cost, privacy risk) or fully device-based (weak planning/reflection). Prior device–cloud hybrids are open-loop—cloud issues instructions to the device with no feedback path—requiring frequent screenshot uploads for verification, which leaks private screen content and increases latency. There is no closed-loop, privacy-preserving architecture that can recover from execution failures without heavy cloud involvement.

Method

EcoAgent uses three specialized agents in a feedback loop:

  1. Cloud-Based Planning Agent with Dual-ReACT: extends the ReACT framework with two nested levels. Global ReACT decomposes the user instruction into subgoals given the initial screen \(S_0\). Local ReACT maps each subgoal to a concrete action step \(ST_t\) and an explicit expected outcome \(EX_t\): $\(P_0 = \text{GlReACT}(\text{Ins}, S_0) = \{\text{LoReACT}(ST_1, EX_1), \ldots, \text{LoReACT}(ST_t, EX_t)\}\)$ When a step fails, a Memory + Reflection module uses stored screen trajectory text to replan: \(P_n = \text{Reflection}(\text{Ins}, P_{n-1}, \text{Memory})\).

  2. Device-Based Execution Agent: a fine-tuned MSLM (ShowUI-2B or OS-Atlas-Pro-4B) performs UI grounding and executes actions \(O_t = \text{EA}(S_t, ST_t)\).

  3. Device-Based Observation Agent (Qwen2-VL-2B, not GUI-fine-tuned): verifies each step \(R_t = \text{OA}(S_{t+1}, EX_t)\) by comparing the post-action screen against the expectation generated by Dual-ReACT. Its Pre-Understanding Module compresses the screenshot \(S_{t+1}\) into a 50–150 token text description \(T_{t+1} = \text{PreUnderstand}(S_{t+1})\) (vs. >1400 tokens for a raw image), which is stored in memory and transmitted to the cloud only on failure.

Key Contributions

  • Closed-loop device–cloud multi-agent architecture that enables device-to-cloud feedback without transmitting raw screenshots.
  • Dual-ReACT: a two-stage (global + local) reasoning paradigm that generates executable steps paired with verifiable expectations, enabling lightweight on-device verification.
  • Pre-Understanding Module: screenshot-to-text compression that reduces cloud token consumption by ~10–28× per screen while mitigating privacy leakage.
  • DeleteText() action extension to the mobile action space for robust text-editing tasks.

Results

  • Task success rate (AndroidWorld, 116 tasks): EcoAgent (OS-Atlas) 27.6% vs. M3A (cloud dual-agent, GPT-4o×2) 28.4% — near parity despite far less cloud use.
  • Operational cost vs. AppAgent: −76% MLLM calls (MC), −79% MLLM tokens (MT).
  • Operational cost vs. M3A: −89% MC, −96% MT.
  • Operational cost vs. UGround-V1-2B (open-loop hybrid): 6.5× fewer MC, 14× fewer MT.
  • Latency: EcoAgent (ShowUI) 3.9 s/step vs. M3A 15.3 s, UGround 18.2 s, AppAgent 7.1 s; only device-only agents (ShowUI 1.2 s, V-Droid 3.0 s) are faster.
  • Ablation: Execution-only SR = 7.0%/4.3%; adding Planning Agent (Dual-ReACT) → 15.5%/19.0%; adding Observation Agent → 25.6%/27.6%, with only modest additional MC/MT increase.

Limitations

  • Overall SR (27.6%) remains well below the best device-based agent V-Droid (59.5%) and cloud multi-agent Agent S2 (54.3%), suggesting the closed-loop design does not yet close the gap for the hardest tasks.
  • Evaluation is limited to a single benchmark (AndroidWorld); generalization to other mobile platforms or task distributions is untested.
  • The Planning Agent still requires a capable cloud MLLM (GPT-4o), so fully offline operation is not yet feasible.
  • Pre-Understanding Module relies on the Observation Agent's ability to produce faithful textual summaries; information loss in compression may degrade replanning quality.
  • Full on-device deployment remains an open challenge pending hardware and model-efficiency advances.

Relevance to Harnesses / Meta-Harnesses

EcoAgent exemplifies a meta-harness pattern: a controlling orchestrator (Planning Agent) that dynamically dispatches specialized sub-agents, collects their structured feedback, and invokes reflection/replanning loops — the same architectural skeleton used in software test harnesses and pipeline orchestrators. The Dual-ReACT mechanism operationalizes the "generate expectation → verify → replan" loop that is central to self-correcting harness design. The Pre-Understanding Module shows a concrete strategy for inter-agent communication bandwidth reduction (image → structured text) that is directly applicable to any multi-agent harness where raw modality data would create bottlenecks. The closed-loop feedback architecture, ablation-validated component decomposition, and cost/latency instrumentation make this paper a useful reference for harness engineers designing heterogeneous agent pipelines with mixed compute tiers.