Dynamic Coordination Strategy Selection for Enterprise Multi-Agent Systems¶

🕒 Published (v1): 2026-05-30 16:43 UTC · Source: Arxiv · link

Why this paper was selected

Evidence-based guidance for choosing consensus, debate, or single-agent coordination

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper evaluates whether dynamically selecting coordination strategy (consensus, debate, synthesis, single-agent) by problem class outperforms a fixed global policy for enterprise multi-agent LLM systems. Across 1,440 judged outputs (30 tasks × 5 problem classes × 4 strategy conditions × 3 replications × 4 model arms), strict pre-registered exact-winner prediction fails, but a weaker near-best routing claim holds: the predicted strategy stays within 0.10 quality-score points of the observed best in every pre-registered (model, problem-class) pair. The practical upshot is that problem-class routing is a useful calibrated default, not a deterministic law.

Problem¶

Enterprise multi-agent deployments expose multiple coordination primitives (consensus, debate, synthesis, single-agent) but lack empirical guidance on which to use per task class. Using a mismatched pattern adds latency, cost, verbosity, or citation errors without improving quality. Prior work has shown multi-agent coordination helps in some settings but has not provided a routing policy grounded in a controlled, pre-registered cross-industry evaluation.

Method¶

A frozen 2D matrix of 30 enterprise tasks spans six industries (including Vietnamese-domain banking/insurance) and five pre-registered problem classes (PC1: high-uncertainty risk; PC2: conflicting-objective tradeoff; PC3: novel design synthesis; PC4: structured compliance verification; PC5: ambiguous-requirement clarification). Each task runs under four coordination conditions × three replications per cell × three pre-registered model arms (Qwen local, Sonnet, Gemma via OpenRouter) plus an auxiliary OpenAI arm—yielding 1,440 judged outputs. A fixed Sonnet rubric scores all outputs. Analysis uses Friedman permutation tests (\(\alpha = .05\), 5,000 permutations, fixed seed 20260527), paired bootstrap CIs (5,000 reps) for predicted-vs-best-non-predicted gaps, exact signed-rank sign-flip enumeration with Bonferroni adjustment, and Kendall's \(W = \chi^2_F / (m(k-1))\) for H2 language-stratification. A GPT-5.1 second-judge sensitivity study re-scores all 480 rep-1 outputs to check judge-dependence.

Key Contributions¶

Pre-registered evaluation design for multi-agent coordination routing across 5 problem classes × 6 enterprise industries, with exact statistical scaffold and public seed (20260527).
Empirical refutation of strict exact-winner routing: bootstrap CIs include zero or favor the alternative in all but one cell (Qwen PC3, margin +0.025).
Validation of a 0.10-tolerance near-best routing heuristic: all \(\Delta < 0.10\) across all pre-registered (model, problem-class) pairs and replicated under independent GPT-5.1 judge.
Discovery that PC4 (structured compliance verification) is a coordination-harm exception: citation overreach (100% of 62 material harm rows), requirement misses (52%), and verbosity bloat (37%) explain why all four arms select single-agent as exact best.
PC2 subtype split: adversarial decisions → debate; balanced operating posture → consensus or synthesis, driven by metric-accuracy loss rather than citation overreach.
Null result on H2: Vietnamese vs. English industry strata show no reliable difference in strategy-ranking concordance (\(\bar{W}_{\text{VI}}=0.204\), \(\bar{W}_{\text{EN}}=0.201\); signed-rank \(p=.85\), bootstrap 95% CI \([-0.069, 0.070]\)).

Results¶

Near-best routing (H1 revised): Predicted strategy within 0.10 points of best observed condition in 20/20 (model arm, problem-class) pairs across pre-registered + auxiliary arms; all \(\Delta < 0.10\).
Formal cell tests: Friedman detects strategy effects in 9/15 pre-registered cells. Only Qwen PC3 shows predicted strategy exceeding best non-predicted (by 0.025, 95% CI [0.003, 0.048])—below the pre-registered 0.10 winner threshold.
PC4 scores (single-agent vs. consensus): Gemma 0.9532 vs. 0.8571; Qwen 0.9196 vs. 0.8447; Sonnet 0.8624 vs. 0.7990; OpenAI 0.8932 vs. 0.8551.
PC4 coordination-harm coding (62 material rows): citation overreach 62/62 (100%); requirement miss 32/62 (52%); verbosity \(\geq 1.5\times\) baseline 23/62 (37%).
H2: Mean \(W=0.204\) (VI) vs. \(0.201\) (EN); \(p=.85\); no evidence of language-stratified coordination differences.
H3 cross-model winner agreement: Exact winner matches in only PC4 (unanimous: single-agent); mixed in all other problem classes.
Second judge (GPT-5.1): Pearson \(r=0.40\) with Sonnet scores; near-best routing holds in 13/15 pairs; both misses are PC4 cells where consensus trails single-agent by 0.16–0.25, reinforcing the PC4 exception.

Limitations¶

Fixed Sonnet rubric judges all outputs; cross-model comparisons are measurements under a Sonnet-judge regime, not model-independent ground truth (though GPT-5.1 second-judge partially addresses this).
Task corpus is a curated 30-task benchmark across six industries; results are policy evidence for selected enterprise families, not universal proof.
Strategy implementations are specific operationalizations (fixed prompts, agent counts, arbiter roles); different implementations could shift exact winner identity.
Formal test scaffold is dependency-light (no external stats library); authors flag it should be expanded for conference submission.
OpenAI arm added post-hoc as auxiliary; not part of original confirmatory design, and GPT-5.1 second-judge shows self-preference on the auxiliary OpenAI arm.
H1 near-best interpretation is post-hoc; the pre-registered strict exact-winner criterion failed, narrowing the confirmatory claim.

Relevance to Harnesses / Meta-Harnesses¶

This paper is directly relevant as an empirical study of meta-level routing logic within multi-agent harnesses: it operationalizes and tests a dispatch table that selects coordination strategy by problem class, which is the core decision a meta-harness must make when routing tasks to sub-agent topologies. The finding that near-best routing holds under a 0.10 tolerance band provides a concrete calibration target for harness designers choosing between consensus, debate, synthesis, and single-agent nodes. The PC4 coordination-harm result (citation overreach from aggregation in deterministic rule-check tasks) is a direct warning for harnesses that default to multi-agent patterns on all task types—deterministic verification tasks should bypass coordination layers. The pre-registered strategy mapping (Table, Section 3.2) doubles as a reference routing policy transferable to any harness that classifies incoming tasks by problem class.