Skip to content

CHORUS: An Agentic Framework for Generating Realistic Deliberation Data

🕒 Published (v1): 2026-04-22 15:01 UTC · Source: Arxiv · link

Why this paper was selected

Agentic framework generates large-scale realistic deliberation data at scale

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CHORUS is a multi-agent orchestration framework that generates realistic online deliberation data by deploying LLM-powered actors with structured personas on live web platforms. Each actor is an autonomous agent with personal posting/action history, and participation timing is governed by independent Poisson processes rather than LLM self-scheduling. A pilot on the DELIBERATE civic platform with 10 actors over 20 minutes received mean expert scores of 4.6 (content realism), 4.3 (analytical utility), and 4.1 (discussion coherence) on a 5-point Likert scale.

Problem

Large-scale deliberation data for online discourse analysis is scarce due to platform access restrictions, privacy constraints, and inconsistent dataset quality. Existing multi-agent simulation frameworks target goal-directed tasks or broad social dynamics (e.g., OASIS, Smallville) and lack: (1) integration with real interactive platforms, (2) principled temporal modeling of heterogeneous participation rates, and (3) sustained, contextually coherent multi-turn deliberative discourse.

Method

CHORUS instantiates \(N\) actors \(\mathcal{A} = \{a_1, \ldots, a_N\}\), each governed by an autonomous LLM agent with a structured persona \(\rho_i\) encoding biography, communication style, core beliefs, and engagement patterns. Each actor maintains two private histories: \(H_i^{\text{post}}\) (prior posts) and \(H_i^{\text{action}}\) (prior votes), preventing redundant engagement.

Temporal model: Participation timing is decoupled from the LLM. Each actor's posting and voting events are sampled from independent Poisson processes with actor-specific rates \(\lambda_i^{\text{post}}\) and \(\lambda_i^{\text{action}}\). A global priority queue \(Q\) orders all pending events; the simulation loop extracts the earliest event and dispatches either POST(a_i) or ACTION(a_i), then schedules the next event for that actor by adding a new Poisson draw to the current time.

POST procedure: The agent draws \(v \sim U(0,1)\); if \(v \leq p_i^{\text{reply}}\) it selects a target post from the shared history and generates a reply (agree/disagree, persona-consistent); otherwise it generates a new comment. In both cases, the agent may optionally invoke a shared tool suite \(\mathcal{T}\) (post submission, history retrieval, upvote/downvote, web search for the Expert archetype).

ACTION procedure: The agent selects \(M\) candidate posts from the aggregated history and applies a Bernoulli filter: a vote on candidate \(c\) is executed only if \(u \sim U(0,1) > \theta_i^{\text{action}}\), where \(\theta_i^{\text{action}}\) is an actor-specific selectivity threshold.

The framework was deployed on DELIBERATE (a civic AI platform) with 10 actors across four archetypes (Casual User, Expert, Advocate, Skeptic), all powered by Claude Sonnet 4.5, at under €2 API cost for a 20-minute simulation.

Key Contributions

  • CHORUS orchestration framework: multi-actor LLM simulation with structured, behaviorally consistent personas for deliberative discourse generation.
  • Poisson process-based temporal model: decouples participation timing from LLM priors, using per-actor \((\lambda_i^{\text{post}}, \lambda_i^{\text{action}})\) rates to approximate heterogeneous real-user engagement.
  • Structured tool suite integration: enables actor interaction with live web platforms (post, vote, web search), translating generative outputs into concrete platform actions.
  • Live deployment on DELIBERATE civic platform; open-source release of framework and pilot discussion data.
  • Expert evaluation protocol across content realism, discussion coherence, and analytical utility.

Results

  • Content Realism: \(\mu = 4.6 / 5.0\) (30 expert evaluators; QA practitioners, linguists, AI researchers).
  • Analytical Utility: \(\mu = 4.3 / 5.0\) — synthetic data yielded thematic NLP outputs consistent with genuine climate discourse.
  • Discussion Coherence: \(\mu = 4.1 / 5.0\) — lowest score, attributed to emergent multi-party dynamics (topic drift, implicit context) being difficult to replicate through persona conditioning alone.
  • Posting rates fluctuated between 3–13 posts/minute; voting actions 6–25/minute over \(T = 20\) minutes — no periodic structure, approximating bursty real-platform patterns.
  • Archetype behavioral differentiation confirmed: Skeptic 2 produced more replies than new comments (\(p^{\text{reply}} = 0.75\)); Expert produced exclusively new comments; Advocates showed highest posting volume.

Limitations

  • No ablation studies isolating individual component contributions (persona grounding, Poisson temporal model, tool usage) against simpler baselines (e.g., single-LLM monolithic generation).
  • Evaluation is small-scale: \(N=10\) actors, \(T=20\) minutes, one topic (climate change), one platform.
  • Tool invocation frequency and its effect on content quality are unquantified.
  • Sensitivity to model substitution (only Claude Sonnet 4.5 tested) is unexamined.
  • Expert evaluators are all affiliated with the DELIBERATE project, introducing potential evaluator bias.
  • LLM censorship may suppress "negative" persona traits (low agreeableness, high neuroticism), reducing behavioral diversity — a known limitation acknowledged from prior work but not directly addressed.

Relevance to Harnesses / Meta-Harnesses

CHORUS is a concrete multi-agent harness whose control loop (priority queue, Poisson event dispatch, per-agent history, tool invocation) is a reusable orchestration pattern directly relevant to harness design. The key architectural decision — separating when agents act (Poisson scheduler, outside the LLM) from what they do (persona-conditioned LLM generation) — is a generalizable harness principle that avoids model-prior bias in scheduling. The structured tool suite abstraction (\(\mathcal{T}\) as a platform adapter layer) parallels the tool-routing and action-translation responsibilities common in production harnesses. For meta-harness researchers, CHORUS also demonstrates the cost and quality tradeoffs of running sustained multi-actor simulations (\(N=10\), \(T=20\) min, <€2) on current frontier models.