Skip to content

MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MASTER is a data augmentation framework that uses multi-agent simulations of classroom pedagogy—error correction, debate, and analogical reasoning—to generate high-quality instruction-tuning data. The resulting 19K-sample BOOST-QA dataset, when used for LoRA fine-tuning, consistently outperforms original data and competing augmentation baselines across math, coding, and general reasoning benchmarks.

Problem

High-quality instruction fine-tuning data is scarce and expensive to produce. Existing synthetic augmentation methods rely on manually crafted prompts or simple lexical transforms, lack coherent interaction mechanisms, and fail to replicate the cognitive diversity present in real-world problem-solving, leading to poor generalization on complex tasks.

Method

MASTER instantiates a multi-agent classroom simulator (MACLASS) with teacher and student agents, each assigned role- and phase-specific prompts to prevent role drift. Three augmentation modules operate on source datasets (Orca-Math-200K, ProcQA, OpenHermes2.5):

  1. Error Correction (ME): A weak student model (Qwen2.5-0.5B, T=0.8) generates a plausibly incorrect answer; a teacher (Qwen2.5-14B, T=0.2) diagnoses the error; the student then self-corrects. This injects structured perturbation into the gradient landscape (Eq. 1: ∇θ L_aug ≈ ∇θ L(θ) + γ·E_{δy}[∇θ ℓ(f_θ(x), δy)]).
  2. Debate (DB): Three student agents (two Qwen2.5-7B at T=0.6; one Qwen2.5-14B at T=0.2 as summarizer) argue across 1–2 rounds before a consensus answer, diversifying gradient signals (Eq. 2).
  3. Analogical Expansion (EP): After a first-round Q&A, cosine similarity via all-MiniLM-L6-v2 retrieves a semantically similar question from the residual pool; the student solves both, and the pair is concatenated in ShareGPT format, training joint probability p(y1,y2|x1,x2) (Eq. 4).

All outputs are formatted in ShareGPT multi-turn dialogue and verified with Qwen2.5-32B-Instruct; 4.1% of samples contained reasoning errors.

Key Contributions

  • MASTER framework: pedagogically grounded multi-agent data augmentation covering error correction, debate, and analogical reasoning.
  • BOOST-QA: 19K instruction-tuning samples augmented from 19K original samples drawn from three domains (math, code, general).
  • Gradient-level analysis characterizing how each scenario regularizes the loss landscape.
  • Empirical demonstration that all three scenarios are necessary—single- or dual-scenario ablations underperform even the original data on average.

Results

  • BOOST-QA vs. Ori-Data (LLaMA3-8B): average across 7 benchmarks improves from 45.98 → 51.26; HumanEval 39.02 → 50.61; MMLU-PRO-MATH 13.55 → 27.39.
  • BOOST-QA vs. Ori-Data (Qwen2.5-7B): average 37.27 → 60.64; ARC 20.90 → 68.52; MMLU 24.05 → 50.12.
  • Vs. baselines (LLaMA3-8B, Table 3): BOOST-QA average 51.26 outperforms Ori (45.98), RandomAug (38.77), SpellingAug (29.21), TAGCOS (47.93), CoT-fine (41.11).
  • Complex multiple-choice tasks (Figure 2): average improvement of 15.50% over ori-data across 8 MC benchmarks, peak +31.46% on TAL-SCQ5K-MC.
  • Ablation: Full 3-scenario MASTER (avg 51.26) > any 2-scenario combination (best DB&EP: 39.02) > any single scenario (all < 38.40) > Ori-Data (45.98) — two-scenario combos actually fall below the original baseline on average.

Limitations

  • All augmentation agents are from the Qwen2.5-Instruct series; teacher quality is capped by Qwen2.5-14B, which may propagate its biases.
  • 4.1% procedural error rate in augmented data is detected but not fully corrected.
  • Only 7B-class base models evaluated; scaling behavior to larger or RLHF-tuned models is untested.
  • Analogical retrieval depends on sentence-level embedding similarity (all-MiniLM-L6-v2), which may not capture deep structural analogy.
  • The framework requires running multiple LLM inference calls per sample, making augmentation computationally expensive relative to simple baselines.
  • No analysis of failure modes when the weak student model (0.5B) produces syntactically broken rather than meaningfully wrong answers.

Relevance to Agentic AI / LLM Agents

MASTER demonstrates a concrete productive use of multi-agent interaction beyond task execution: agents with heterogeneous capability levels collaboratively generate training data, effectively using agent behavior to improve the base models that power future agents. The structured role assignment and turn-management protocols address known failure modes of LLM-based multi-agent systems (role drift, topic drift, repetition), providing a reusable design pattern. For researchers building agentic systems, this work contributes a scalable synthetic data strategy that improves reasoning and multi-step problem solving—capabilities directly bottlenecking agent performance—without requiring human-annotated traces. It also operationalizes pedagogy-inspired interaction protocols as an agent coordination primitive.