Aligned Agents, Biased Swarm: Measuring Bias Amplification in Multi-Agent Systems

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Bias amplifies through MAS pipelines even when base models are individually aligned

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Multi-agent systems (MAS) do not dilute social bias—they amplify it. Across all tested architectures, topologies, and model combinations, bias (measured via Gini coefficient on demographic preference distributions) monotonically increases as information cascades through agent chains. Even individually neutral agents produce systemic polarization, with a "Trigger Vulnerability" showing that injecting purely objective context dramatically accelerates this amplification.

Problem

Single-LLM alignment research treats bias as a static property of model weights, measurable via single-turn benchmarks. This misses emergent, system-level bias that arises when aligned individual agents interact in structured MAS workflows. Existing benchmarks use binary prompts that modern LLMs are fine-tuned to answer safely, masking latent preferences. The fundamental question—whether MAS architectures mitigate or amplify residual biases—was empirically unexamined.

Method

The authors model a MAS as a directed acyclic graph \(G = (V, E)\) where each agent \(A_j\) receives an aggregated context \(C_j = \mathcal{A}(Q, \{S_m\}_{m \in P(j)})\) and outputs a state \(S_j = (p_j, R_j)\): a probability distribution over options and a textual rationale. Bias is measured as deviation from the uniform distribution \(p_u = (1/k, \ldots, 1/k)\) using the Gini coefficient:

\[G(p) = \frac{\sum_{l=1}^{k}(2l - k - 1)p_{(l)}}{k-1}\]

They introduce Discrim-Eval-Open, reformulating Anthropic's Discrim-Eval binary benchmark into a three-option forced-choice format: each scenario presents three demographically distinct candidates (varying age, gender, race), compelling comparative judgments that expose latent preferences modern LLMs suppress in binary settings. The benchmark contains 70 scenarios × 3 options = 210 unique profiles balanced across age, gender, and race. Bias amplification is tracked via layer-wise (\(\alpha_i = \bar{B}_i / \bar{B}_{i-1}\)) and total (\(\beta_i = \bar{B}_i / \bar{B}_0\), operationalized as "relative Gini") amplification factors. Experiments span: (1) sequential chains with identical/persona-varied/function-varied agents; (2) three topologies—Spindle, Parallel, Fully-Connected; (3) iterated fully-connected units; (4) heterogeneous model mixtures; (5) RAG-style neutral context injection. Eight LLMs were tested: DeepSeek-V3, DeepSeek-R1, Step-1, GPT-4o, GPT-4o-mini, GLM-4v, Qwen-Max, Gemini-1.5-Pro.

Key Contributions

• Reframing bias as an emergent systemic property of MAS interaction structure rather than a static deficiency of individual model weights.
• Discrim-Eval-Open: a three-option forced-comparative benchmark (210 profiles; age, gender, race attributes) designed to bypass performative neutrality of aligned LLMs.
• Distributional bias metrics suite: Gini coefficient, variance, entropy, and relative Gini for measuring polarization across agentic chains.
• Empirical mapping showing all common MAS design strategies (persona diversity, functional roles, complex topologies, model heterogeneity) fail to prevent and often amplify bias.
• Identification of Trigger Vulnerability: injecting neutral, factual external text (as in RAG) triggers severe polarization spikes even in otherwise stable systems.

Results

• Sequential chains (baseline): Relative Gini increases monotonically from L1→L4 across all 8 LLMs; confirmed universal amplification (\(\beta > 1\) at every layer).
• Persona specialization (Doctor, Lawyer, Engineer, Merchant): amplification trend continues upward despite role diversity; no statistically meaningful mitigation.
• Functional roles (Judger→Analyst→Reflector→Summarizer): Reflector agent causes a temporary partial dip at L3 for some models, but amplification resumes by L4.
• Topologies (Spindle, Parallel, Fully-Connected): all show progressive amplification across L1–L4; Fully-Connected produces the most pronounced amplification due to richer information exchange.
• System depth (iterated FC units I0→I4): steepest and most sustained amplification observed; deeper MAS are strictly less robust.
• Model heterogeneity (GPT-4o-mini only vs. DeepSeek-R1 only vs. hybrid): all three amplify; hybrid relative Gini at I4 is intermediate (≈1.44) between homogeneous systems (≈2.04 and ≈1.20), but still unambiguously amplifying.
• Emergent demographic preferences: system-level preferences for younger individuals, females, and Black candidates emerge even when individual agents are nominally neutral.
• Trigger Vulnerability: introducing neutral factual context (simulating RAG retrieval) causes a dramatic spike in polarization, demonstrating extreme fragility of system-level robustness.

Limitations

• Study is restricted to baseline/foundational MAS mechanics; does not evaluate production-scale agentic frameworks (e.g., full tool-use pipelines, long-horizon task systems).
• Discrim-Eval-Open covers only demographic bias (age, gender, race); other bias dimensions (political, economic, geographic) are not assessed.
• Paper text is truncated, so full quantitative results for the Trigger Vulnerability and bias preference analysis (Figure 8) are not available in the provided excerpt.
• Experiments use synchronous, static agent chains; dynamic, asynchronous, or memory-augmented MAS architectures are not evaluated.
• Causal mechanism for amplification is characterized qualitatively (sycophancy, confirmation bias) but not formally proven; the DAG model is descriptive rather than predictive.
• Limited to text-based scenarios; multimodal or tool-use agents are out of scope.

Relevance to Agentic AI / LLM Agents

This paper establishes a critical safety baseline for anyone designing or deploying LLM-based multi-agent systems: individual-level alignment does not transfer to system-level ethical robustness. The finding that architectural sophistication—role specialization, complex topologies, model diversity—consistently exacerbates rather than mitigates bias has direct implications for agentic pipeline design, where chained reasoning and inter-agent delegation are standard. The Trigger Vulnerability is particularly alarming for RAG-augmented agents, since injecting grounded factual context—typically a safety feature—can serve as an amplification catalyst. The formal DAG framework and distributional metrics (Gini, entropy) provide a reusable toolkit for auditing bias propagation in arbitrary agentic workflows.