Skip to content

Youtu-Agent: Scaling Agent Productivity with Automated Generation and Hybrid Policy Optimization

🕒 Published (v1): 2025-12-31 04:17 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Youtu-Agent is a modular LLM agent framework from Tencent that automates agent construction (tool synthesis + YAML config generation) and adds continuous optimization via a training-free in-context learning module and a scalable RL training pipeline. It achieves SOTA on WebWalkerQA (71.47%) and GAIA (72.8%) using open-weight models, while its RL module trains Qwen2.5-7B to 45% on AIME 2024 from a 10% baseline with 40% faster iteration.

Problem

Building capable LLM agents requires expensive manual effort: tool integration, prompt engineering, and environment wiring create a high barrier to entry. Once deployed, agents are static—adapting them requires either brittle manual prompt tuning or full fine-tuning (SFT/RL), which is data-hungry, computationally costly, and prone to training instability (entropy explosion) in long-horizon tasks.

Method

The framework is organized into three decoupled layers—Environment (backends: browser via Playwright, shell, E2B sandbox), Tools (atomic/composite/MCP), and Agent (perceive–reason–act loop with a Context Manager for token pruning)—all declared via YAML configs.

Automated generation operates in two modes: - Workflow mode: deterministic 4-stage pipeline (intent clarification → tool retrieval/synthesis → prompt engineering → config assembly); synthesizes missing Python tools including unit tests via LLM code generation. - Meta-Agent mode: an Architect Agent treats generation capabilities (search_tool, create_tool, ask_user, create_agent_config) as callable tools, enabling multi-turn clarification and flexible tool synthesis.

Continuous optimization has two components: - Agent Practice: Training-free GRPO—runs parallel rollouts on ~100 samples, has an LLM evaluator distill a "semantic group advantage" (textual diff of successful vs. failed trajectories), injects accumulated experience as "textual LoRA" at inference time; zero gradient updates required. - Agent RL: Integrates with VeRL via Agent-Lightning; uses RESTful API wrapping + Ray-based concurrency for distributed rollout, hierarchical timeout logic for scale-up to 128 GPUs, and training stabilization via invalid-call filtering, no batch shuffling, reduced off-policy iterations, and corrected turn-level GRPO advantage estimation.

Key Contributions

  • YAML-based modular architecture decoupling Environment/Tools/Agent for automated synthesis and reuse
  • Dual-paradigm agent generation (Workflow + Meta-Agent) with automated Python tool synthesis (>81% executability)
  • AgentGen-80 benchmark (80 task descriptions) for evaluating automated agent generation quality
  • Training-free GRPO (Agent Practice): in-context experience accumulation without gradient updates, ~\(18 cost vs. ~\)10K for RL baselines
  • Agent RL pipeline: 40% iteration speedup, stable 128-GPU scaling, entropy-explosion mitigations

Results

  • WebWalkerQA (680 Qs): 71.47% pass@1 with open-weight DeepSeek-V3 models
  • GAIA text-only (466 Qs): 72.8% pass@1
  • AgentGen-80 automated generation: Workflow mode 100% config validity / 81.25% tool executability / 65% task completion; Meta-Agent mode 98.75% / 82.50% / 68.75%
  • Agent Practice (DeepSeek-V3.1-Terminus, Mean@32): AIME 2024: 80.0% → 82.7% (+2.7%); AIME 2025: 67.9% → 73.3% (+5.4%), using 100 samples, ~\(18 cost (vs. ~\)10K for ReTool/SimpleTIR on Qwen2.5-32B)
  • Agent RL (Qwen2.5-7B): AIME 2024: 10% → 45% (+35%); AIME 2025: 9% → 31% (+22%); search QA gains of +8–21% across TriviaQA, PopQA, NaturalQuestions, MuSiQue, HotpotQA, Bamboogle, 2WikiMultiHop; 40% training iteration speedup vs. Agent-Lightning v0.2.2

Limitations

  • AgentGen-80 is self-constructed with only 80 tasks; no independently validated benchmark exists for automated agent generation quality
  • GAIA evaluation restricted to text-only subset due to use of text-only models, limiting comparability with multimodal baselines
  • Tool executability ceiling of ~82%: ~18% of synthesized tools fail, constraining end-to-end task completion
  • Task completion rates (65–68.75%) on AgentGen-80 leave substantial room for failure in automated deployments
  • Agent Practice gains are measured on a single domain (math); generalization to other task types is not demonstrated
  • Training-free GRPO variant without ground truth underperforms the supervised variant, limiting applicability where task outcomes are ambiguous

Relevance to Harnesses / Meta-Harnesses

Youtu-Agent is directly a meta-harness: its Meta-Agent mode instantiates a higher-order agent whose sole job is to generate, configure, and wire together other agents—the defining pattern of a meta-harness. The YAML-based configuration system with automated tool synthesis operationalizes the harness-as-code concept, enabling programmatic construction of pipelines rather than hand-coded scaffolding. The dual optimization loop (inference-time experience accumulation + RL training) represents a self-improving harness architecture where the meta-level continuously refines the object-level agents it spawns. For researchers tracking harness design, the Training-free GRPO mechanism is particularly notable as a lightweight alternative to RL-based harness adaptation that requires no access to model weights.