Skip to content

Co-EPG: A Framework for Co-Evolution of Planning and Grounding in Autonomous GUI Agents

🕒 Published (v1): 2025-11-13 03:41 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Co-EPG is a self-iterative training framework that jointly optimizes a decoupled planning model (Ï€) and grounding model (Ï•) for GUI task automation through a mutual reinforcement loop: the grounding model provides reward signals to train the planner via GRPO, while the improved planner generates higher-quality data to fine-tune the grounder. After three iterations using only benchmark data, Co-EPG reaches SOTA on Multimodal-Mind2Web (58.4% Step SR) and AndroidControl (83.1% Step Acc).

Problem

Existing GUI agent architectures treat planning and grounding as independently optimized components, failing to exploit cross-model synergies. They also over-depend on large external synthetic datasets (e.g., AGUVIS-7B uses 283,500 labeled steps), which introduces noise and limits data efficiency. No prior work had established a co-evolution paradigm for decoupled planning-and-grounding architectures.

Method

Co-EPG uses a P-G dual-model architecture where a planning model π (Qwen2.5-VL, 7B or 3B) outputs a textual plan, action type, and action value, and a separate grounding model ϕ (72B) consumes the plan plus screenshot to predict click coordinates. Training alternates between two steps per iteration k:

  1. Iterative Training: π_k is refined via GRPO with rewards from C-DREM (Confidence-based Dynamic Reward Ensemble Mechanism). C-DREM aggregates rewards from multiple grounding models (trained ϕ_k plus Qwen2.5-VL-72B/32B ensembles) using a weighted sum where weights combine a static prior σ_j (higher for the trained grounder) and a dynamic confidence score c_j derived from token log-likelihood of the predicted coordinate. The final reward ri is zero if action type or value is wrong, otherwise equals the plan reward (binary grounding accuracy). ϕ_k is then SFT-fine-tuned on high-quality data distilled from the planning rollouts.

  2. Data Enhancement: A Planner pool Π and Verifier pool Φ (open-source VLMs + latest {π′k, π′) generate and filter plans, retaining only those the grounding verifier successfully executes, producing the enhanced dataset D_k for the next iteration.}} and {ϕ_k, ϕ_{k-1}

Key Contributions

  • Co-EPG framework: closed-loop co-evolution of decoupled planning and grounding models via GRPO + SFT without external data
  • C-DREM: ensemble reward mechanism with confidence-based dynamic weighting across multiple grounding models, reducing reward noise and accelerating GRPO convergence
  • Data Enhancement loop: self-evolving data pipeline that increases both plan purity (+8.84% over 3 iterations) and diversity (+4 plans/task) without external annotation
  • Demonstrated 2.42% data usage relative to AGUVIS-7B while surpassing it on Multimodal-Mind2Web

Results

  • Multimodal-Mind2Web: Co-EPG-Web-7B achieves 58.4% avg Step SR (Cross-Task 66.3%, Cross-Website 62.3%, Cross-Domain 59.3%), surpassing AGUVIS-7B (57.2%) and Explorer-7B (54.3%)
  • Multimodal-Mind2Web 3B: Co-EPG-Web-3B achieves 51.4% avg Step SR, outperforming Explorer-4B (49.8%) by ~1.6%
  • AndroidControl: Co-EPG-Mob-7B achieves 83.1% avg Step Acc (High 74.2%, Low 92.0%), +1.4% over UI-TARS-7B (81.7%); Co-EPG-Mob-3B achieves 81.8% vs. InfiGUI-R1-3B (81.6%)
  • Ablation — P-G decoupling: +3.4% over end-to-end fine-tuned model (53.5% vs. 50.1%)
  • Ablation — C-DREM: full C-DREM (+1.91%) > prior weights only (+0.66%) > average ensemble (+0.51%) > single reward model (56.5% baseline)
  • Ablation — GRPO: removing GRPO from iterative loop consistently degrades performance at each iteration stage for both 3B and 7B models

Limitations

  • Three iterations of co-evolution are evaluated; it is unclear whether gains plateau or reverse beyond M3
  • The grounding model (72B) used for reward in C-DREM is very large; computational cost per training cycle is not reported in detail
  • Self-iterative loop relies on the quality of the initial dataset D0; bootstrapping from weak initial data could stall improvement
  • Evaluated only on Multimodal-Mind2Web and AndroidControl; generalization to other GUI environments (desktop, other OSes) is untested
  • Data Enhancement pool retains only the two most recent model versions, potentially discarding useful diversity from earlier iterations

Relevance to Agentic AI / LLM Agents

Co-EPG demonstrates a scalable self-improvement paradigm for tool-using agents that avoids dependence on large external supervision — a central challenge in building autonomous agents. The co-evolution of a high-level planner and low-level executor via reward-coupled RL is directly relevant to hierarchical agent architectures where policy decomposition is necessary for complex, multi-step tasks. The C-DREM reward ensemble addresses a fundamental problem in LLM agent training: reward signal reliability when no ground-truth execution feedback is directly available, analogous to challenges in tool-call grounding for code or API agents. This work advances the case that modular agent architectures can be trained end-to-end through closed-loop RL without requiring massive human-annotated trajectory datasets.