Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents¶

🕒 Published (v1): 2026-07-16 15:08 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Digital Pantheon is a multi-agent framework for simulating parliamentary coalition formation by combining SFT+DPO ideological alignment with per-party RAG to overcome the neutrality bias of RLHF-trained LLMs. It operationalizes on the 2019 Flemish election using a hub-and-spoke negotiation arena (party agents + formateur) and introduces MILT and CIS as auditable explainability layers that trace every clause back to its manifesto origin. Across three independent runs, manifesto-anchored provisions reliably match the historically adopted agreement while hallucinated content does not.

Problem¶

RLHF-trained LLMs are tuned toward neutrality and helpfulness, making sustained partisan behavior impossible: prompt-based personas degrade reasoning and lack domain grounding, while multi-agent debate amplifies conformity and dominant-party bias. Prior political simulators either predict terminal outcomes without simulating deliberation, or simulate deliberation without attributing clauses to verifiable ideological origins—leaving the negotiation process a black box.

Method¶

A three-stage pipeline decouples factual grounding from persona injection:

ASH Chunking: Manifesto PDFs are partitioned by an Agentic–Structural Hybrid procedure combining a deterministic Hard-Gate (regex-detected structural boundaries) and a probabilistic Soft-Gate (Gemma3-27B binary same-topic classifier on trailing 800-character context windows at temperature 0.1).
Two-Stage Alignment: Each party's adapter is trained first with SFT (B=2, E=5 epochs) on Chosen responses to instill a first-person "Method Actor" persona, then with DPO (B=1, E=3, $\beta=0.1$ KL penalty, lr=$5\times10^{-6}$) to push probability mass away from neutral/opposing responses. Steps are scaled as $I = \frac{N}{B} \times E$ to equalize alignment depth across manifestos of varying length. Base model is Gemma-3 27B quantized to 4-bit via QLoRA on Apple MLX.
Per-Party RAG: Isolated ChromaDB collections (one per party) with paraphrase-multilingual-MiniLM-L12-v2 dense vectors; top-$k$ chunks (cosine similarity $< 0.51$, $k\leq 5$) are injected as hard ideological constraints at inference.
Hub-and-Spoke Negotiation Arena: Party agents (spokes) present standpoints; a formateur (hub, instantiated from the largest party's adapter but system-prompted as a neutral broker) drafts and refines an agreement over four rounds across all 22 Flemish government policy dossiers. Banzhaf power indices are passed to the formateur to enforce realistic power dynamics. Adapter hot-swapping keeps base weights static in unified memory.
MILT Explainability: Qwen3.6-27B NLI classifier traces every $L_3$ (final agreement) clause backward through $L_1$ (agents' standpoints) to $L_0$ (manifesto chunks), classifying into five provenance states: Direct Lineage, Diluted Lineage, Synthesized Lineage, Pipeline Artifact, and Orphan (hallucination).
Coalition Influence Score (CIS): $$\text{CIS}(\pi) = \sum_{p:\,\pi(p)=\pi} w(\tau(p)), \quad w = \begin{cases} 3 & \tau = \text{Direct} \\ 2 & \tau = \text{Diluted} \\ 1 & \tau = \text{Synthesized} \\ 0 & \tau \in \{\text{Pipeline, Orphan}\} \end{cases}$$
Real-World Grounding ($L_\text{Real}$): Every $L_3$ provision graded against the historically adopted 2019 agreement as Present / Partially Present / Absent using the same Qwen3.6 NLI pipeline; realization index $G = (n_\text{present} + 0.5\, n_\text{partial})/N$ per dossier.

Key Contributions¶

Behaviorally steered LLM party agents that maintain partisan consistency under extended negotiation via parameter-level SFT+DPO (not prompt engineering)
ASH chunking procedure combining deterministic structural rules with LLM-based topic coherence checks for auditable, provenance-preserving manifesto segmentation
MILT: a temporal directional graph over $L_0/L_1/L_3$ that classifies every clause into five provenance states, separating valid negotiation dynamics from systemic pipeline errors
CIS: an ordinal influence metric that aggregates traceable MILT contributions while zero-weighting hallucinated content
External validation showing manifesto-anchored lineage predicts real-world materialization; hallucinated provisions (Orphan/Pipeline) are empirically found to mostly fail to appear in the actual adopted agreement
Adapter hot-swapping design enabling all major party agents on consumer hardware

Results¶

Stable winner and ranking across all 3 independent simulations: N-VA $>$ CD&V $>$ Open Vld (by CIS)
Mean ~876 classified provenance points per run; $N = 2{,}629$ total points across 3 runs
Provenance-bearing provisions (Direct + Diluted + Synthesized) account for nearly all "Present" and most "Partial" grounding judgements against the 2019 historical agreement
Orphan and Pipeline Artifact classes are overwhelmingly graded "Absent" in the real-world grounding pass — an empirical finding, not an artifact of the evaluation design
The ordering of MILT states by materialization rate is stable across every simulation run and evaluator pass
Applied across all 22 policy dossiers of the Flemish government portfolio

Limitations¶

Four-round negotiation horizon is a design choice justified by qualitative saturation in preliminary runs, not by a formal convergence criterion
Only three simulation runs; no statistical power analysis or sensitivity test on the number of rounds
Single electoral scenario (2019 Flemish); generalizability to other parliamentary systems, languages, or election cycles is untested
CIS weights (3/2/1) are ordinal encodings, not calibrated magnitudes; any monotonically decreasing weighting over the three states preserves the ranking
Formateur behavior is sensitive to fine-tuning: a non-fine-tuned base formateur exhibited bias toward moderate parties even when given Banzhaf indices
Real-world grounding uses LLM-as-judge (Qwen3.6), inheriting opacity concerns the paper itself cites against prior work

Relevance to Harnesses / Meta-Harnesses¶

Digital Pantheon is itself a multi-stage meta-harness: it chains a document-processing sub-harness (ASH chunking), an alignment pipeline (SFT→DPO per party), a retrieval harness (per-party isolated ChromaDB), and a negotiation orchestration layer (hub-and-spoke arena with adapter hot-swapping) into a single end-to-end system. The MILT evaluation suite functions as an audit harness running atop the simulation harness, tracing provenance through the pipeline's intermediate outputs ($L_0 \to L_1 \to L_3$) rather than only the terminal result—directly addressing the "outcome-only black box" failure mode common in agent pipelines. The design pattern of decoupling persona injection (DPO adapter) from factual grounding (RAG) and orchestration (formateur) is a reusable architectural principle for any harness that must coordinate multiple behaviorally differentiated agents. For researchers tracking harnesses, this paper demonstrates how provenance auditing and structured orchestration can make emergent multi-agent behavior interpretable at scale.