ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Non-autoregressive multi-turn agentic interaction generation improves efficiency

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ToolACE-MT is a non-autoregressive framework for generating high-quality multi-turn agentic dialogue data without relying on costly turn-by-turn multi-agent simulation. It generates full conversational trajectories via coarse initialization, iterative mask-and-fill refinement, and offline verification. Models trained on ToolACE-MT data outperform those trained on autoregressive multi-agent simulation (MAS) data across multiple tool-use benchmarks.

Problem

Constructing multi-turn agentic training data via multi-agent simulation (MAS) is expensive: each turn is generated autoregressively, task complexity is hard to control, and the assistant lacks global trajectory context, causing factual inconsistencies, poor tool-call accuracy, and limited solvability in long-horizon tasks.

Method

ToolACE-MT adapts the non-autoregressive translation (NAT) and masked diffusion paradigms to the dialogue turn level, generating trajectories in three stages:

1. Coarse-Grained Initialization: Given a sampled tool pool, subtasks \((u_1, \ldots, u_m)\) are generated with metadata (required tools, step counts). Sub-trajectories \(C_t\) are composed sequentially (tool calls and outputs in parallel within each sub-trajectory), then concatenated into \(C = C_0 \cup \cdots \cup C_m\). This prioritizes structural completeness over semantic correctness.

2. Iterative Refinement: Two alternating pass types:

3. Complexity Injection (mask-and-extend): inserts realistic phenomena—clarification turns, tool-awareness exchanges, error simulation, non-function-calling turns—by masking a turn and extending the trajectory with new content \(f_{\text{LLM}}(\sigma, (\ldots, X, \ldots))\).

4. Reasonability Refinement (mask-and-fill): randomly masks non-adjacent turns and regenerates them; an LLM judger accepts or rejects each regeneration. Selection probability for a turn decreases after it is chosen, promoting coverage. Up to 5 refinement rounds per instance; 1–3 complexity types injected.

5. Offline Verification: Hybrid rule-based (format compliance, executability, repetition, ID-consistency hallucination) and model-based (LLM sub-question experts for semantic coherence and complex hallucinations) filtering.

Key Contributions

• Non-autoregressive iterative generation pipeline (ToolACE-MT) for multi-turn agentic dialogue synthesis, decoupling trajectory structure from turn-by-turn LLM rollout.
• Complexity injection via mask-and-extend and reasonability refinement via mask-and-fill, enabling explicit control over dialogue complexity and budget-scalable generation.
• Demonstrated efficiency and quality gains: ToolACE-MT data yields higher coherence (EnR 50.71 vs. 43.60, SS 68.34 vs. 65.23) and diversity (Dist-3 0.357 vs. 0.319) than MAS data at equal dataset size (8,000 instances each).

Results

• BFCL-v3 Multi-Turn: ToolACE-MT (Llama3.1-8B-Inst base) achieves 40.25% overall multi-turn accuracy vs. 31.38% for MAS, 12.50% for Llama3.1-70B-Inst, and 9.25% for the base model; outperforms Gemini-2.5-Pro-Preview (34.62%) and DeepSeek-V3 (29.87%).
• BFCL-v3 Single-Turn (Non-Live): 84.94% vs. 80.29% for MAS, preserving base model capability.
• ACEBench Agent (Process Accuracy): ToolACE-MT 34.0% vs. MAS 15.0%; End-to-End Accuracy 8.4% vs. 6.7%.
• τ-Bench (Avg.): ToolACE-MT 20.6% vs. MAS 16.0%.
• Ablation: Removing Offline Verification drops BFCL-v3 overall by ~2.4%; further removing Iterative Refinement drops it to 20.88% (from 40.25%).
• Generalization confirmed on Qwen2.5-Instruct (0.5B–7B) and Qwen3-8B backbones.

Limitations

• Agent end-to-end accuracy remains very low for 8B-scale models on ACEBench, suggesting the framework does not fully close the capability gap for smaller models on hard agentic tasks.
• The base Llama3.1-8B-Inst outperforms all fine-tuned models on τ-Bench Airline, indicating domain-specific anomalies that the framework does not resolve.
• Generation still depends on a capable LLM (GPT-4o-2024-11-20) for all stages; the framework reduces turn count but not the per-call LLM cost.
• Live single-turn performance improves less than non-live because models trained with rich multi-turn data develop a bias toward requesting clarification on ambiguous queries.
• Evaluation is limited to English benchmarks; cross-lingual generalization is not assessed.

Relevance to Agentic AI / LLM Agents

ToolACE-MT directly addresses the data bottleneck in training tool-using LLM agents: replacing expensive online MAS rollouts with a controllable offline generation pipeline that produces higher-quality multi-turn trajectories at lower cost. By introducing global trajectory awareness at initialization time, it overcomes a core weakness of autoregressive data synthesis—the inability of an assistant agent to plan coherently without seeing the full task. The mask-and-fill iterative refinement is a novel design pattern transferable to other structured agentic data generation problems (e.g., planning traces, tool-chain supervision). For researchers tracking agentic SFT data pipelines, this paper offers a concrete alternative to MAS and connects to the broader trend of applying discrete diffusion ideas to structured sequence generation beyond machine translation.