Skip to content

Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence

🕒 Published (v1): 2025-01-01 · Source: ICLR · Venue: ICLR 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

IoA is a distributed multi-agent framework modeled on the Internet: heterogeneous third-party agents register on a central server, dynamically form nested sub-teams via instant-messaging-style group chats, and coordinate via a finite-state machine governing conversation flow. With only basic ReAct agents, IoA reaches 40.0% overall on the GAIA benchmark, surpassing prior MAS baselines, and wins 66–76% of head-to-head comparisons against AutoGPT and Open Interpreter on open-ended tasks.

Problem

Existing multi-agent frameworks suffer from three compounding limitations: (1) ecosystem isolation — agents outside the framework's own ecosystem cannot be integrated; (2) single-device simulation — real distributed deployments are not supported; (3) rigid pipelines — communication topology, team membership, and state transitions are hard-coded rather than dynamically negotiated at runtime.

Method

IoA introduces a three-layer client–server architecture (Interaction / Data / Foundation) with four interlocking mechanisms:

  1. Agent Registration & Discovery — agents register capability descriptions \(d_i\) on a central server; any agent can call search_client(L_d) → P(C) to find collaborators matching desired characteristics \(L_d\) via semantic matching.
  2. Autonomous Nested Team Formation — a root group chat \(g_0\) spawns sub-group chats \(g_l\) recursively as sub-tasks require additional expertise, yielding a tree \(h: G \to \mathcal{P}(G)\). Nesting reduces communication channels: \(c_\text{nested}(g) \leq c_\text{full}(g) = \frac{|g|(|g|-1)}{2}\).
  3. Finite-State Machine Conversation Control — group chat state \(M = (S, \Sigma, \delta, s_0, F)\) with states \(S = \{s_d, s_s, s_a, s_p, s_c\}\) (discussion, sync/async task assignment, pause-and-trigger, conclusion). Each agent's LLM autonomously computes \((s_{t+1}, c_{t+1}) = f_\text{LLM}(M_t, s_t)\) to select next state and next speaker.
  4. Task Assignment & Execution — tasks \(t = (d_t, S_t)\) are allocated synchronously (chat paused until completion) or asynchronously (parallel execution), with an Agent Integration Block providing a uniform run: String → TaskID interface for third-party agents.

Key Contributions

  • An agent integration protocol enabling heterogeneous third-party agents (AutoGPT, Open Interpreter, etc.) running on different devices to join a shared collaboration space.
  • An instant-messaging-inspired architecture with server-side agent registry, group-chat routing, and client-side wrappers.
  • Autonomous nested team formation that reduces communication complexity compared to flat fully-connected topologies.
  • A Speech-Act-Theory-grounded FSM for group-chat flow control with LLM-driven state transitions and speaker selection.
  • Demonstrated cross-domain applicability without task-specific prompt tuning across GAIA, open-ended instruction, embodied AI, and RAG benchmarks.

Results

  • GAIA benchmark (validation set): IoA (4 basic ReAct agents) achieves 40.0% overall vs. AutoGen 39.39%, FRIDAY 34.55%, GPT-4+Plugins 14.60%, AutoGPT-4 4.85%.
  • Open-ended instruction benchmark (153 tasks, 4 categories): IoA wins 66.7% vs. AutoGPT and 76.5% vs. Open Interpreter overall; strongest in Math (83.3% vs. both) and Coding (83.3% vs. AutoGPT).
  • GPT-3.5-based IoA on RAG QA achieves performance close to or exceeding GPT-4 baselines and surpasses prior MAS on the same task.

Limitations

  • Central server is a single point of coordination; scalability and fault tolerance under large agent counts are not evaluated.
  • Semantic matching for agent discovery quality and latency are not ablated.
  • FSM state transitions depend entirely on LLM judgment; failure modes (infinite loops, stuck states) are not quantified.
  • Experiments use GPT-4-1106-preview; generalization to smaller or open-weight LLMs is not assessed in the main paper.
  • Cost analysis and nested-team formation precision are deferred to appendices, not the main results.

Relevance to Harnesses / Meta-Harnesses

IoA is a meta-harness in the precise sense: it provides a protocol layer and orchestration runtime that wraps arbitrary third-party agents without modifying them, coordinates their discovery and teaming dynamically, and governs execution flow via an FSM — all concerns that define a meta-harness rather than a task-specific agent. The agent registration/discovery mechanism directly instantiates the "capability registry" pattern central to composable harness design. The nested team formation with recursive sub-group spawning mirrors the hierarchical task decomposition seen in agentic harnesses like AutoGen, but externalizes the topology to runtime negotiation rather than compile-time wiring, which is a key design axis for harness extensibility.