Skip to content

Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

OSU/CMU; Agent Data Protocol unifies heterogeneous agent fine-tuning datasets at scale

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Agent Data Protocol (ADP) is a standardized "interlingua" representation that unifies heterogeneous agent training datasets into a single schema, reducing dataset-to-framework conversion effort from \(O(D \times A)\) to \(O(D + A)\). By converting 13 existing datasets (1.3M trajectories) into ADP and fine-tuning Qwen2.5-Coder models, the authors achieve ~20% average improvement over base models across coding, web browsing, tool use, and research benchmarks without domain-specific tuning.

Problem

Large-scale supervised fine-tuning (SFT) of LLM agents is rare despite abundant data because existing agent datasets are fragmented across incompatible formats, action spaces, and observation structures. Each dataset–framework pair requires bespoke engineering, making cross-dataset integration brittle and preventing the community from combining complementary data sources at scale.

Method

ADP is implemented as typed Pydantic schemas defining a Trajectory object containing an alternating sequence of Actions (three types: APIAction with function/kwargs, CodeAction with language/content, MessageAction with content) and Observations (two types: TextObservation with source/content, WebObservation with HTML, accessibility tree, URL, viewport, optional screenshot). A three-stage pipeline handles conversion: 1. Raw→ADP: per-dataset converter maps dataset-specific structures to the ADP schema once. 2. ADP→SFT: per-framework converter (one per agent harness—OpenHands, SWE-Agent, AgentLab) translates ADP into the framework's action/observation space, system prompts, and conversation format. 3. Quality Assurance: automated validation checks tool call format, ≥80% function-call/thought pairing, and proper conversation termination.

Base models are Qwen2.5-Coder-Instruct (7B/14B/32B), fine-tuned via LLaMA-Factory on the unified 1.3M-trajectory ADP Dataset V1, with subsampling to balance domain representation.

Key Contributions

  • ADP schema: a minimal, expressive Pydantic-based representation covering API/tool use, code execution, web browsing, and general agentic workflows.
  • Converters for 13 existing datasets (AgentInstruct, CodeActInstruct, SWE-Gym, Mind2Web, SWE-smith, NNetNav, etc.) into ADP, and converters from ADP to three agent frameworks.
  • ADP Dataset V1: 1.3M agent training trajectories, the largest publicly available agent SFT dataset.
  • Cross-dataset analysis revealing action-type distributions, trajectory length variance (1–26.8 turns), and high function-thought coverage (≥90% in most datasets).
  • Demonstrated cross-task transfer: training on the full ADP mixture substantially outperforms training on any individual dataset.

Results

  • SWE-Bench Verified (SWE-Agent, 7B): 0.4% → 20.2% (+19.8%); surpasses Claude 3 Opus (15.8%) and SWE-smith-tuned baseline (15.2%).
  • SWE-Bench Verified (OpenHands, 7B): 2.8% → 20.4% (+17.6%); surpasses SWE-Gym-tuned baseline (10.6%).
  • WebArena (AgentLab, 7B): 4.5% → 21.0% (+16.5%).
  • AgentBench OS (OpenHands, 7B): 3.5% → 27.1% (+23.6%); surpasses AgentInstruct-tuned baseline (17.4%).
  • GAIA (OpenHands, 7B): 7.3% → 9.1% (+1.8%).
  • Average gain of ~20% over corresponding base models across all benchmarks and model sizes (7B, 14B, 32B).
  • ADP-trained 7B models are competitive with or superior to prior SOTA from similarly-sized models on each benchmark without domain-specific tuning.

Limitations

  • GAIA gains are modest (+1.8%), suggesting the current data mixture does not fully cover complex multi-modal, multi-step reasoning required by that benchmark.
  • Evaluation is limited to three agent frameworks (OpenHands, SWE-Agent, AgentLab); generalization to other architectures is not demonstrated.
  • Data quality in recorded rollouts is bounded by the capability of the baseline agent used to generate trajectories, limiting trajectory diversity and difficulty ceiling.
  • Subsampling strategy for balancing large datasets (e.g., Orca AgentInstruct at 1.046M) is heuristic; optimal mixture weights are not ablated exhaustively.
  • The paper does not evaluate instruction-following or safety properties of ADP-tuned models.

Relevance to Agentic AI / LLM Agents

ADP directly addresses the infrastructure bottleneck that has kept large-scale agent SFT an exception rather than standard practice: format heterogeneity across datasets. By establishing a community-wide interlingua, it enables researchers to pool diverse trajectory data (web, code, SE, tool use) without bespoke engineering per dataset–framework pair, mirroring how standardized pre-training corpora accelerated LLM scaling. The demonstrated cross-task transfer—where mixed training systematically outperforms single-domain training—has direct implications for building generalist agents capable of tool use, browsing, and software engineering without specialization. For teams building open-source agent pipelines, the linear \(O(D+A)\) conversion cost model and the 1.3M-trajectory public dataset lower the bar for reproducing and extending SFT-based agent research.