A superpersuasive autonomous policy debating system¶

🕒 Published (v1): 2025-11-22 00:45 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

DeepDebater is a hierarchical multi-agent system that autonomously executes full eight-speech competitive policy debates, beating human-authored strategies 85% of the time in simulated rounds. It pipelines specialized LLM-agent workflows—each handling a distinct argumentative sub-task—over a 3M-card evidence corpus, and adds a live animated presentation layer via TTS and talking-head synthesis.

Problem¶

Prior AI debate systems (e.g., IBM Project Debater) targeted simplified, evidence-light formats for lay audiences, sidestepping the game-theoretic, iterative, and rigorously evidence-grounded nature of real competitive policy debate. No system existed that could handle the full lifecycle of a two-team, eight-speech round including cross-examinations, multi-pronged negative strategies, and line-by-line rebuttals.

Method¶

DeepDebater chains a sequence of role-specialized multi-agent workflows built on AG2/AutoGen, each following the same core pattern: generate, structured-generate (Pydantic-validated outputs), retrieve (BM25/DuckSearch over DuckDB-indexed OpenDebateEvidence), and critically review—iterating until a Reviewer agent is satisfied. The pipeline proceeds sequentially through the entire debate round (1AC → 1NC → 2AC → ... → 2AR), with each workflow receiving the full accumulated transcript as context. Sub-workflows are further decomposed: e.g., the 1AC stage spawns separate agent teams for Plantext, Harms, Inherency, Solvency, and each Advantage's Uniqueness/Link/Internal-Link/Impact chain. An independent Judge Agent (Gemini, Claude, or GPT-4.1) reads the complete transcript and renders a Reason for Decision. Finalized transcripts are voiced via GPT-4o-mini-TTS and animated with EchoMimic V1.

Key Contributions¶

Hierarchical multi-agent framework that decomposes an entire competitive policy debate lifecycle into discrete, evidence-grounded, role-based agent workflows.
Live end-to-end presentation pipeline: transcript → TTS audio → lip-synced talking-head portrait video, rendered in real time during the round.
Hybrid human–AI operation mode allowing humans to substitute for any speech or full team.
Empirical demonstration that the system outperforms human-authored debate cases as rated by expert coaches and as judged by autonomous LLM judges.
Open-source release with all code, generated transcripts, audio, and video.

Results¶

Expert coach evaluation (5 coaches with 10+ years, 3 advantages each, 1–5 scale):

Metric	Our System	Human-Authored
Quality	4.32 ± 0.31	3.65 ± 0.52
Factuality	4.45 ± 0.25	3.98 ± 0.23
Faithfulness	4.81 ± 0.19	4.05 ± 0.48

Simulated round performance (20 rounds, autonomous Gemini judge):
System as Negative vs. human 1AC: 90% win rate (9/10)
System as Affirmative vs. human 1NC: 80% win rate (8/10)
Overall: 85% win rate
Cross-judge robustness on same 20 transcripts: Claude judge gives 80% win rate (\(\Delta = -5\) pp, \(\kappa = 0.75\)); GPT-4.1 gives 83% (\(\Delta = -2\) pp, \(\kappa = 0.89\)) vs. Gemini baseline.

Limitations¶

Evaluation uses an autonomous LLM judge rather than human judges for the bulk of round assessments, introducing potential circularity (the system was designed partly with LLM judges in mind).
Evidence retrieval uses BM25 keyword search only; the authors explicitly note that dense embeddings over OpenDebateEvidence would substantially improve quality but are too expensive.
Human evaluation is small-scale: 5 coaches, 6 advantages total (3 system, 3 human), on a single topic.
No evaluation against live, practicing human debaters in real tournament conditions.
The win-rate metric conflates judge model preferences with true debate quality.

Relevance to Harnesses / Meta-Harnesses¶

DeepDebater is a concrete, domain-specific instance of a sequential multi-stage harness: it pipelines LLM agent workflows in strict order, with each stage's output becoming the next stage's context—exactly the pattern meta-harnesses generalize. The repeatable generate → structured-output → retrieve → review loop within each workflow is a reusable harness primitive that could be extracted and applied to other long-horizon creative or analytical tasks. For researchers building harnesses, the system demonstrates how Pydantic-enforced structured outputs and BM25 retrieval loops can be wired into a workflow engine (AG2) to enforce schema compliance across many chained agents. It also surfaces a key meta-harness design question: how to propagate accumulated state (the growing transcript) as shared context across heterogeneous sub-agents without blowing the context window.