Meta-Agent: From Task Descriptions to Verified Multi-Agent Systems¶

🕒 Published (v1): 2026-05-24 19:38 UTC · Source: Arxiv · link

Why this paper was selected

Auto-generates verified multi-agent systems from natural-language task descriptions

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Meta-Agent is a two-phase framework that automatically constructs and executes task-specific multi-agent systems from natural-language descriptions, treating the agent system itself as a synthesize-and-verify artifact. It introduces construction-time and execution-time verification loops with typed error attribution (local/upstream/structural) to prevent cascading failures in long-horizon workflows. On six benchmarks, it achieves an average score of 82.7 vs. AFlow's 80.3, with the largest gains on reasoning-heavy tasks.

Problem¶

Existing multi-agent frameworks rely on fixed or manually designed agent structures that assume task decomposition is known a priori. Small errors at intermediate stages propagate silently through agent interactions, while self-reflection and post-hoc correction mechanisms are insufficient for long-horizon workflows where failures compound. There is no principled mechanism for automatically generating, grounding, and verifying both the structure and behavior of a multi-agent system from underspecified task descriptions.

Method¶

Meta-Agent operates in two phases over a DAG $G = (V, E)$ of agent specifications $\{\sigma_1, \ldots, \sigma_n\}$:

Construction phase: A task planner parses the natural-language task description $T$ into a structured intent representation, then decomposes it into a DAG of agent specifications. Each node $\sigma_i$ carries an explicit input/output contract, a role description, a tool interface, and verification criteria. A web-search module grounds each specification with retrieved external knowledge (APIs, documents). A code-generation module produces system prompts and tool configurations. Construction-time verification then applies two checks per agent: (1) static validation (code structure, interface completeness, instantiability) and (2) behavioral simulation via a verifier model on representative inputs. Failures return typed signals $f \in F$ (specification mismatch, grounding failure, contract violation), routing regeneration to the specific upstream stage responsible.

Execution phase: A coordinator dispatches agents in topological order; outputs $y_i$ are gated by per-agent criteria $C_i$ (schema, behavioral assertions, forbidden patterns) before propagation: $y_i \in C_i$. The unified verification loop is: $$\text{Generate} \to \text{Verify} \to \text{Attribute} \to \text{Refine}$$ A three-level error attribution mechanism classifies failures at agent $a_i$ as: - Local: $a_i$ produced wrong output despite correct inputs → local retry with verifier feedback. - Upstream: failure originates in a dependency → re-execute the responsible upstream agent. - Structural: flawed task decomposition (e.g., mismatched I/O contracts) → partial re-construction of the agent subgraph.

Recovery cost is thus proportional to error locality.

Key Contributions¶

Introduces the meta-agent paradigm: an agent that constructs other agents, treating the multi-agent system as a per-task synthesized artifact rather than a fixed architecture.
A unified two-phase framework integrating planning, web-grounded knowledge retrieval, code generation, and verification at both construction and execution time.
A typed three-level error attribution mechanism (local / upstream / structural) enabling targeted, cost-proportional recovery without full pipeline restarts.
Empirical demonstration that construction-time verification is the single most impactful component (−7.1 DROP points when ablated).

Results¶

Average across 6 benchmarks (GPT-4o-mini): Meta-Agent 82.7 vs. AFlow 80.3 (+2.4), CoT-SC 76.0 (+6.7), ADAS 67.2 (+15.5).
MATH: +13.4 points over AFlow (69.6 vs. 56.2) — largest single-benchmark gain.
HumanEval: 96.0 vs. AFlow 94.7 (+1.3), best overall.
MBPP: 84.6 vs. AFlow 83.4 (+1.2).
GSM8K: 93.7 vs. AFlow 93.5 (+0.2).
DROP: 82.7 vs. AFlow 80.6 (+2.1).
HotpotQA: 69.5 vs. AFlow 73.5 (−4.0) — only benchmark where Meta-Agent trails.
Ablation on DROP: Removing verification: −7.1; removing API research: −5.5; removing planning: −3.5; removing prompt analysis: −2.4.

Limitations¶

Trails AFlow on HotpotQA (69.5 vs. 73.5), suggesting the framework may underperform on pure reading comprehension/retrieval tasks where multi-agent decomposition adds overhead rather than benefit.
Verification introduces additional computational cost at every intermediate step; the paper reports this is empirically offset but does not provide wall-clock or token-cost measurements.
Single-run scores are reported, limiting statistical reliability of the comparisons.
DAG structure and agent specifications are regenerated per task from scratch with no template reuse, which may be expensive at inference time for tasks where fixed structures would suffice.
Evaluated only with GPT-4o-mini as the primary executor (Claude Sonnet 4.6 results mentioned but details truncated in the provided text).

Relevance to Harnesses / Meta-Harnesses¶

Meta-Agent is a direct instantiation of the meta-harness concept: a system that, given a task description, synthesizes and orchestrates a purpose-built multi-agent pipeline rather than executing a fixed workflow. The construction phase — where a planner generates a DAG of agent specs, retrieves grounding knowledge, produces code, and verifies the whole structure before any execution — is architecturally equivalent to a harness-compiler. The typed error attribution and closed-loop Generate→Verify→Attribute→Refine pattern is a concrete formalization of the retry/escalation logic that meta-harnesses need to implement reliably. For practitioners building harnesses, the ablation evidence that construction-time verification contributes more to robustness than any other single component (+7.1 points) is directly actionable design guidance.