Skip to content

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles

🕒 Published (v1): 2025-12-25 15:02 UTC · Source: Arxiv · Venue: ICML 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MoRAgent proposes a parameter-efficient fine-tuning (PEFT) framework for LLM agents that decomposes agent capabilities into three roles—reasoner, executor, summarizer—each served by a dedicated group of LoRA adapters. A rule-based role-aware gate and learnable token-aware routers dynamically route token representations to the appropriate role-specific LoRAs within a single model. This achieves competitive or superior performance to full-parameter SFT while adding only ~0.16–0.36B trainable parameters on top of 1–4B base models.

Problem

Directly applying LoRA to agent fine-tuning performs significantly worse than full-parameter SFT because agent tasks require simultaneously learning reasoning, tool-calling, and summarization—capacities that are difficult to compress into a single low-rank update. Full-parameter SFT, in contrast, is computationally expensive and degrades the base model's general capabilities.

Method

Agent capability is decomposed into three roles following the ReAct Reason+Action paradigm: Reasoner (interprets query, decides next role from trajectory), Executor (selects function name and parameters), Summarizer (distills conversation history into user response). Each role is implemented as a group of LoRAs: one shared LoRA plus several routed LoRAs selected by a Top-K token-aware router. A hard role-aware gate enforces mutual exclusivity—each token position is processed by exactly one role's LoRA block (all others zero out). The training loss combines cross-entropy, an auxiliary load-balancing loss (from Switch Transformers), and an orthogonal loss that pushes routed LoRAs to learn linearly independent directions, reducing redundancy. A multi-role data pipeline uses GPT-4o to complete missing reasoning/summary steps in existing tool-use datasets and DeepSeek-V3 for quality filtering.

Key Contributions

  • Decomposition of agent capabilities into three distinct roles (reasoner, executor, summarizer) grounded in the ReAct paradigm.
  • Mixture-of-Roles (MoR) architecture: role-specific LoRA groups with a rule-based role-aware gate and learnable token-aware Top-K routers, enforcing strict per-token role exclusivity.
  • Orthogonal loss between routed LoRAs to maximize feature diversity and minimize parameter redundancy.
  • Multi-role data generation pipeline: GPT-4o for content completion + DeepSeek-V3 for reliability verification + manual correction for executor errors.

Results

  • StableToolBench (DFS setting, Llama3.2-1B-Instruct + 0.16B trainable): +44.5% pass rate, +25.2% win rate vs. base; MoRAgent-Phi (3.82B+0.36B) surpasses ToolLLaMA-v2-7B in average pass rate (58.7% vs. 60.6% for a 7B model).
  • BFCL leaderboard (avg. accuracy): Llama3.2-1B+0.16B → 77.6% (+50.1% over base); Phi3.5-mini+0.36B → 82.1% (+17.9%); openPangu-Embedded-1B+0.20B → 77.2% (+53.4%).
  • GSM8K / MATH (Qwen2.5-1.5B-Coder + 0.27B): +13.8% on GSM8K, +12.0% on MATH vs. best baseline.
  • Ablation (BFCL, Llama3.2-1B): MoR (0.16B) achieves 77.6% avg. accuracy vs. LoRA 65.5%, DoRA 66.8%, full SFT (1.24B) 76.0%; balance + orthogonal losses together add +5.3%.
  • With only 1K training samples, MoRAgent still achieves +27.4% over base, demonstrating data efficiency.

Limitations

  • Evaluated only on small-to-medium base models (1B–4B range); scaling behavior on larger models (e.g., 70B) is not demonstrated.
  • The role decomposition is fixed at three roles; more complex agent tasks requiring additional capability facets are not addressed.
  • Data pipeline relies on GPT-4o and DeepSeek-V3 for completion and verification, introducing dependency on proprietary models and potential noise propagation.
  • Inference requires the role-aware gate to operate sequentially (reasoner → executor or summarizer), which may not parallelize as cleanly as vanilla LoRA.

Relevance to Agentic AI / LLM Agents

MoRAgent directly targets a practical bottleneck in deploying capable open-source agents: full-parameter SFT is expensive and destroys general capabilities, while naive LoRA underperforms on multi-step tool use. By structuring PEFT around the functional decomposition inherent to the ReAct paradigm, this work bridges the gap between single-LoRA PEFT and full SFT without adding separate models (unlike multi-LLM approaches such as α-UMi). The role-gating mechanism is conceptually analogous to mixture-of-experts routing applied at the agent-workflow level, offering a principled way to specialize capacity for reasoning vs. execution within a single inference pass—relevant to researchers studying efficient agent fine-tuning, structured tool-use, and LoRA-based specialization.