Skip to content

Autonomous Event-Driven Multi-Agent Orchestration for Enterprise AI at Scale

🕒 Published (v1): 2026-06-18 10:32 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper evaluates two multi-agent orchestration architectures—DAG Plan & Execute and ReAct—across 208 production-derived enterprise scenarios at three organizational scales (Persona: <10 agents, Department: 20–80, Enterprise: 200), and introduces a Task Manager that enables continuous event-driven operation via priority inference, event merging, and preemption. The central finding is that scale, not task complexity, is the dominant failure mode: agent discovery noise degrades performance far more than structural task difficulty. The Task Manager reduces high-priority queue latency by 14–75% and improves related-event correctness by over 20 percentage points at enterprise scale.

Problem

Existing multi-agent systems assume discrete request-response workflows and have not been evaluated under continuous enterprise event streams, large agent registries (hundreds of agents), or task prioritization/merging requirements. Benchmarks like MCP-Bench and Tool Decathlon cover single-agent tool use, not multi-agent orchestration at scale. The gap is that no system studies how orchestration performance degrades as agent registry size grows, or how to schedule, merge, and preempt concurrent event-driven tasks.

Method

Architectures evaluated: - ReAct: Single LLM reasoning-action loop using PydanticAI with parallel A2A tool calls to specialist agents; no upfront planning; handles failures incrementally by continuing the loop. - DAG Plan & Execute: Three-phase system—Planner generates a typed DAG upfront using two-tier agent discovery (lightweight summaries first, full cards only for selected agents); Executor dispatches nodes in parallel batches via A2A, advancing through a quantum budget before hitting a CHECKPOINT; Replanner adapts the DAG on CHECKPOINT, INTERRUPT, or RESUME, performing DAG edits when agents remain sufficient or full rediscovery otherwise.

Task Manager: An OS process-scheduler analog that sits upstream of both architectures. It consumes events from a message broker, uses an LLM to decide whether to create a new task or merge into an existing one (validated against schema constraints before mutating backlog state), and assigns priority. The composite scoring function is: $\(\text{score} = \text{base\_priority} \times (1.0 + \text{time\_bonus}) + \text{progress\_factor}\)$ where \(\text{base\_priority} \in \{1.0, 2.0, 3.0\}\), \(\text{time\_bonus} = \max(0, (\text{task\_age}/\text{mean\_age} - 1.0)) \times w_t\), and \(\text{progress\_factor} = (\text{completed\_nodes}/\text{total\_nodes}) \times w_p\). Preemption only occurs at deterministic stoppage points: DAG checkpoints (before replanning) or ReAct inter-step boundaries (after an agent call returns, before the next LLM step).

Evaluation harness: 200 LLM-powered mock agents conforming to the A2A protocol, each requiring correct entity identification (hard filter) before releasing information. Correctness measured via LLM-as-judge against expected answer fragments.

Key Contributions

  • Empirical evaluation of DAG Plan & Execute vs. ReAct across three enterprise registry scales (208 scenarios, 393 events, 1,051 agent calls)
  • Identification that scale (agent discovery noise), not task complexity, is the dominant orchestration bottleneck
  • A Task Manager architecture for continuous event-driven operation: priority inference, conservative event merging, preemption at deterministic stoppage points
  • A two-tier agent discovery mechanism (summary-first, full-card on-demand) to bound prompt context as registry grows
  • A human-in-the-loop benchmark construction pipeline derived from real enterprise workflows across 7 departments and 19 personas

Results

  • Both architectures achieve 74–98% correctness at Persona scale (<10 agents) but degrade sharply: Department 65–80%, Enterprise 32–83%
  • Simple tasks degrade more than Complex tasks at scale: Simple drops from 90–96% (Persona) to 36–54% (Enterprise); Complex maintains 73–83%—finding 1–3 agents among 200 is harder than finding 4–7 because discovery noise is proportionally more damaging when the target set is smaller
  • ReAct outperforms DAG Plan & Execute on every scenario type at Enterprise scale (200 agents): DAG's plan-interrupt-replan cycle amplifies discovery noise overhead; ReAct handles failures incrementally without triggering full replanning
  • DAG Plan & Execute offers higher precision and structured parallelization at Persona/Department scale due to explicit DAG structure
  • Task Manager reduces high-priority queue latency by 14–75% at enterprise scale
  • Task Manager improves related-event correctness by over 20 percentage points at enterprise scale

Limitations

  • Benchmark dataset (raw source material, workflow specs, implementation code) cannot be released due to enterprise IP constraints; external replication requires rebuilding from scratch using the reported schema
  • Mock agents share a single LLM executor; real-world agents would have independent latency, failure modes, and capability drift not captured here
  • Task Manager's LLM-based event classification (merge vs. create) is evaluated on correctness but not on cost or latency overhead of that classification step
  • Paper is truncated; full enterprise-scale ReAct vs. DAG quantitative breakdown beyond the described trends was not fully available in the provided text
  • The two-tier discovery mechanism does not fully solve the discovery bottleneck at 200 agents—it bounds context but noise remains the primary failure mode

Relevance to Harnesses / Meta-Harnesses

This paper directly models an orchestration meta-harness: the Task Manager + orchestration architecture layer is precisely what a production harness does—ingesting events, scheduling subagents, managing state across concurrent tasks, and enforcing fallback/retry logic. The two-tier agent discovery (summaries for selection, full cards on demand) is a concrete harness design pattern for managing growing tool/agent registries without exploding prompt context. The checkpoint-based preemption and quantum budget mechanism mirror harness control flow constructs (e.g., step budgets, interrupt handlers) that determine when to pause, replan, or hand off. The finding that ReAct is more robust than DAG at enterprise scale because it avoids costly full-replan overhead is directly actionable for anyone designing a meta-harness that must degrade gracefully as the managed agent fleet grows.