Skip to content

ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"

🕒 Published (v1): 2025-08-06 05:04 UTC · Source: Arxiv · Venue: ACL 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ToolGrad inverts the standard tool-use dataset generation paradigm: instead of generating a user query and then searching for a tool-use solution via DFS, it first constructs valid tool-use chains iteratively using a TextGrad-inspired agentic loop, then synthesizes corresponding user queries. This "answer-first" approach achieves a 99.8% annotation pass rate (vs. 63.8% for DFS) while producing more complex chains at lower cost.

Problem

Prior tool-use dataset generation (e.g., ToolBench) generates a user query first, then runs a DFS agent to find a valid tool-use solution. This is inherently failure-prone—exploration offers no guarantee of success—yielding a 63.8% pass rate and wasting agent compute on dead-end trajectories. The core tension is that distilling valid trajectories from complex exploration is expensive by design.

Method

ToolGrad reframes dataset generation as an optimization loop analogous to TextGrad (Yuksekgonul et al., 2024). The dataset \(D = \{(q, W, r)\}\) is built iteratively, where \(W := \{C_1, \ldots, C_n\}\) is a multi-chain API workflow and each chain \(C := \text{API}_1 \to \cdots \to \text{API}_n\).

Each iteration has four modules:

  1. API Proposer (\(\text{LLM}_\text{pr}\)): Given a mini-batch of \(bs=50\) APIs, proposes up to \(m=3\) candidate APIs to extend \(W_t\), using only lightweight API configs (no live calls) for fast filtering.
  2. API Executor (\(\text{LLM}^\mathcal{T}_\text{ex}\)): Each proposed API is executed by a parallel tool-calling LLM agent, producing an execution report \(\text{rep}_i\) with full call history and a success boolean.
  3. API Selector (\(\text{LLM}_\text{sel}\)): Acts as the "gradient" — selects the most valuable API \(j\) and target chain \(k\) from reports: \(j, k = \text{LLM}_\text{sel}(\{\text{rep}_i\}^m, W_t)\).
  4. Workflow Updater: Deterministically appends \(\text{API}_j\) to \(C_k\) (\(W_{t+1} \leftarrow W_t.\text{add}(\text{API}_j, C_k)\)), then synthesizes updated query and response via inverse prediction: \(q_{t+1}, r_{t+1} = \text{LLM}_\text{updater}(W_{t+1})\).

Post-processing samples \(p-n\) negative APIs per example via text-embedding similarity to simulate RAG-like tool retrieval. Generation uses gemini-2.5-flash-lite, 10 iterations/sample, run 500 times to produce ToolGrad-500. Fine-tuning targets Gemma-3 (1B, 4B, 12B) with supervised fine-tuning in one-shot (non-ReAct) tool-call format.

Key Contributions

  • ToolGrad framework: an agentic, answer-first data generation harness with four modular LLM sub-agents orchestrated as an iterative optimization loop
  • ToolGrad-500: 500 synthetic tool-use samples with higher chain complexity than ToolBench, generated at lower cost
  • Open-sourced fine-tuned models: Gemma-3-1B/4B/12B trained on ToolGrad-500, outperforming proprietary LLMs on ToolBench

Results

  • Pass rate: 99.8% (ToolGrad) vs. 63.8% (DFS)
  • Chain complexity: avg. 3.4 ground-truth tool uses per sample (ToolGrad) vs. 2.1 (DFS)
  • Generation cost: LLM calls 63.9 vs. 64.5; tool-call steps 20.0 vs. 34.3 (DFS)
  • ToolBench single-turn scores: ToolGrad-12B (19.6) and ToolGrad-4B (17.6) rank 1st and 2nd, outperforming Claude 4.5 Opus (15.4), GPT-5 base (12.7), and Gemini 2.5 Pro (11.4)
  • BFCL OOD gains over base Gemma: +8.1 (1B), +8.0 (4B), +6.3 (12B); ToolGrad-12B ranks 2nd overall (−0.1 vs. Gemini 2.5 Pro)
  • LLM judge–human rating correlation: \(\rho = 0.88\), \(p < 0.001\) (96 ratings, 2 raters)

Limitations

  • Diversity saturation: at \(>8\)–12 iterations, the agent generates repetitive tool-use patterns; "% of Unique Tool Use" decreases, degrading scaling behavior
  • Sample-count saturation: model performance on BFCL peaks around 500–1k samples and declines with further data scaling under current design
  • Single-turn only: multi-turn and agentic tool-use (BFCL v3/v4) are out of scope; generalization to those settings is unverified
  • Small dataset: ToolGrad-500 contains only 500 samples, limiting large-scale training scenarios
  • Contaminated unsolvable queries: the text mentions "contaminated data with unsolvable queries" as a discussion item (section cut off)

Relevance to Harnesses / Meta-Harnesses

ToolGrad is itself a meta-harness for synthetic data generation: it composes four LLM sub-agents (Proposer, Executor, Selector, Updater) in a deterministic iterative loop, where each agent plays a typed role (filter, execute, judge, synthesize) analogous to forward/backward passes in ML. The "answer-first inversion" is a generalizable harness design principle—restructuring the pipeline to eliminate failure modes rather than filtering them post-hoc, directly relevant to anyone designing robust agentic pipelines. The explicit analogy to TextGrad formalizes multi-agent dataset construction as an optimization loop, offering a transferable abstraction for meta-harness designers. The scaling study surfaces a key meta-harness challenge: synthetic diversity saturation under fixed generation strategies, motivating dynamic harness adaptation as a future direction.