AgensFlow: A Coordination-Policy Substrate for Multi-Agent Systems

🕒 Published (v1): 2026-05-26 08:10 UTC · Source: Arxiv · link

Why this paper was selected

AgensFlow: coordination-policy substrate decoupling agent skills from runtime routing

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgensFlow is an open-source framework that reformulates multi-agent LLM orchestration as an online policy-learning problem under partial observability, making coordination choices—skill protocol, model binding, topology—learnable rather than fixed pipeline decisions. It introduces a folded-signature state abstraction and reliability-aware UCB1 policy graph to route tasks to the best coordination strategy. Evaluated on distributed-systems incident and security-advisory corpora, learned routing outperforms static pipeline baselines on coordination-heavy task classes.

Problem

Static multi-agent pipelines hard-code skill protocols, model-role bindings, and coordination topology, providing no mechanism to adapt to task regime or accumulated operational evidence. One-off benchmark comparisons explore only a small slice of the joint configuration surface (skill × model × topology × evaluation), making it impossible to discover that the optimal configuration varies per task class or to transfer learned priors across domains.

Method

AgensFlow models multi-agent orchestration as a partially observable sequential decision process. The observable state is a folded signature \(s_t = \phi(x_t, h_t, b_t) = (\ell_t, z_t, \beta(\hat{c}_t), \beta(\hat{u}_t), \beta(\hat{r}_t), \beta(\hat{e}_t))\), comprising a regime label \(\ell_t\), a binary 7-bit handoff mask \(z_t\), and four bucketed belief estimates (correctness, uncertainty, contradiction risk, evidence sufficiency). At each routing step the policy selects from a legal action set:

\[a_t \in \mathcal{A}(s_t) = \{\text{invoke}(k,m)\} \cup \{\text{skip:}X\} \cup \{\text{terminate}\}\]

where \((k,m)\) is a skill protocol bound to a model, and \(\text{skip:}X\) omits a still-scheduled topology cell. Action selection uses a reliability-aware UCB1 variant with annealed exploration:

\[\text{score}(s,a) = \bar{r}(s,a) + c_s\sqrt{\frac{\log(N_s+1)}{N_{s,a}}} - \lambda f(s,a)\]

with \(c_s = \max(0.5,\, 1.4 \cdot 2^{-N_s/50})\) and failure penalty weight \(\lambda = 0.5\). Trajectory-level reward composes judged quality with operational cost:

\[r(t) = w_q Q(t) - w_c C(t) - w_\rho \rho(t)\]

with defaults \(w_q=1.0\), \(w_c=0.3\), \(w_\rho=0.15\). Quality \(Q(t)\) is produced by RelativeJudge, a same-class relative ranking module inspired by RULER-style evaluation, extended with cross-judge averaging across multiple model-provider judges, per-axis rubric decomposition (goal achievement, grounding, coordination, recovery), and confidence weighting. The learned object is a persistent, auditable policy graph keyed by folded signature storing visit counts, mean rewards, reward variance, token statistics, and failure rates per \((s,a)\) edge. The framework is structured into six dependency layers: web-search wrappers (L0), schema/regime/skill registry (L1), agents/transport/trace (L2), policy graph/router/LangGraph (L3), reward/RelativeJudge (L4), and pre-flight/governance/reports (L5).

Key Contributions

• Formulation of multi-agent coordination as online policy learning under partial observability, with folded task signatures as the state abstraction enabling value sharing across repeated task trajectories.
• skip:X as a first-class topology action, making trajectory shape (which cells run) a per-task-class learnable variable rather than a fixed pipeline choice.
• Reliability-aware UCB1 with annealed exploration and per-edge failure penalties, separating final reward from repeated execution failures.
• RelativeJudge with cross-judge averaging (multi-provider), per-axis decomposition, and confidence weighting as an auditable reward layer embedded in the system.
• Warm-start policy transfer: saved policy graphs can initialize routing for related domains, reducing exploration cost while preserving adaptation.
• Six-layer open-source substrate with full auditability: operators can inspect learned preferences per regime with visit counts, mean rewards, and failure rates.

Results

• Learned routing vs. fixed pipeline: higher-quality operating point on coordination-heavy task classes (C3 cross-document multi-vendor reasoning, C7 mitigation correctness, C8 cross-vendor pair tasks); ties or narrow trades on procedural classes. Evaluated via 3-judge audit on distributed-systems incident and security-advisory corpora.
• skip:X ablation: no-skip ablation isolates topology compression as a meaningful contributor to substrate quality, confirming that omission of optional cells is a substantive learned behavior and not noise.
• Warm-start transfer: warm-started policy graphs reduce exploration cost while preserving plateau quality compared to cold-start learning, validated in the cross-domain experiment.
• Cross-domain generalization: policy graph trained on one scenario class generalizes across new prompts within the same regime labels, supporting the signature-folding hypothesis.

(No absolute accuracy numbers, latency figures, or exact delta magnitudes are reported in the provided text.)

Limitations

• Belief estimates (\(\hat{c}_t\), \(\hat{u}_t\), \(\hat{r}_t\), \(\hat{e}_t\)) are explicitly described as heuristic in the current release; they are not learned or calibrated.
• Handoff quality belief is tracked but deliberately excluded from \(\phi\) to keep the policy graph compact, creating a known gap between monitored state and decision state.
• Folded-signature granularity introduces a bias-variance tradeoff: coarser bins risk aliasing distinct task states; finer bins require more data before estimates become reliable.
• terminate is not a true policy choice; it is triggered implicitly by evaluator invocation or governance, limiting the policy's control over trajectory length.
• Reward signal depends on LLM judges, which are stochastic; cross-judge averaging mitigates but does not eliminate single-family bias.
• Evaluation is limited to two corpora (distributed-systems incidents and security advisories); generalization to other domains is asserted but not demonstrated beyond same-class cross-domain transfer.

Relevance to Harnesses / Meta-Harnesses

AgensFlow is directly architecturally analogous to a meta-harness: rather than orchestrating a fixed pipeline of agents, it acts as a learned coordination substrate that dynamically selects which agents, skills, models, and topology cells to invoke per task regime—precisely the role a meta-harness plays in managing a fleet of sub-agents. The skip:X mechanism and policy-graph backup are clean primitives for harness designers who need to make topology itself configurable and data-driven rather than hand-coded. The six-layer decomposition (tool wrappers → schema/registry → agents → policy/router → reward/audit → governance) provides a reference architecture for harnesses that need inspectable, auditable control over sub-agent selection and execution. RelativeJudge's integration of the reward signal as a first-class system component (rather than an external evaluator) is a transferable pattern for harnesses that must track quality across repeated runs without privileging a single judge model.