A superpersuasive autonomous policy debating system¶

🕒 Published (v1): 2025-11-22 00:45 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

DeepDebater is a hierarchical multi-agent system that autonomously executes full eight-speech American-style competitive policy debates, using specialized LLM agent workflows grounded in a 3-million-card evidence corpus. It beats human-authored strategies in 85% of simulated rounds as judged by independent LLM judges. The system also supports hybrid human–AI participation and renders debates as lip-synced animated avatars with TTS audio.

Problem¶

Prior autonomous debate systems (notably IBM Project Debater) operated in simplified, shortened formats targeting lay audiences, avoiding the strategic depth, strict evidence-grounding, iterative rebuttal structure, and long-horizon game-theoretic demands of real competitive debate. No system had demonstrated end-to-end autonomous participation in a full, unmodified expert debate format.

Method¶

DeepDebater decomposes policy debate into a sequential pipeline of specialized multi-agent workflows built on the AG2/AutoGen framework with GPT-4.1-mini. Each workflow runs a team of LLM agents that iteratively generate, critique, retrieve evidence, and refine outputs using structured Pydantic schemas for machine-readable inter-agent communication. Evidence is retrieved via BM25 keyword search (DuckSearch) over OpenDebateEvidence, a 3M-card corpus indexed in DuckDB. The pipeline proceeds speech-by-speech in debate order: 1AC construction (plantext → harms/inherency/solvency → advantages), 1NC construction (topicality, disadvantages, counterplans, kritiks, on-case rebuttals), then all subsequent rebuttals and cross-examinations, each conditioned on the full preceding transcript. A separate Judge Agent (Claude or Gemini) reads the complete transcript and renders a Reason for Decision. Finalized transcripts are converted to audio via GPT-4.1-TTS and rendered as talking-head video with EchoMimic V1.

Key Contributions¶

Hierarchical multi-agent framework that decomposes the full lifecycle of competitive policy debate (case construction → multi-pronged negative strategy → rebuttals → cross-examination → adjudication) into role-specialized, self-critiquing agent workflows
End-to-end debate presentation pipeline integrating LLM generation, neural TTS, and audio-driven talking-head animation (EchoMimic V1)
Empirical demonstration via human expert evaluation and simulated round performance that the system produces argumentative artifacts superior to human-authored baselines across quality, factuality, and faithfulness metrics
Open-source release of code, transcripts, audio, and video

Results¶

Expert evaluation (n=5 coaches, 10+ years experience): System scored 4.32 ± 0.31 (quality), 4.45 ± 0.25 (factuality), 4.81 ± 0.19 (faithfulness) vs. human-authored 3.65 ± 0.52, 3.98 ± 0.23, 4.05 ± 0.48 on a 1–5 scale
Simulated rounds (n=20): 85% overall win rate; 90% when playing Negative against human 1ACs, 80% when playing Affirmative against human negative strategies
Cross-judge robustness: Win rate 85% (Gemini), 80% (Claude), 83% (GPT-4.1); pairwise agreement κ=0.75 (Claude vs. Gemini) and κ=0.89 (GPT-4.1 vs. Gemini)

Limitations¶

Evidence corpus (OpenDebateEvidence) ends at 2022; system is prompted to simulate operating in 2022 and cannot "cut new cards"
Evaluation uses LLM-based judges, introducing model-family bias, style-matching bias, and format-familiarity advantages; human study uses only 5 expert coaches with no statistical significance claimed
BM25 retrieval can miss relevant evidence, produce topically skewed results (U.S.-centric corpus), or yield "factually referenced but misleading" claims; no embedding-based retrieval used
Domain-specific: performance tied to U.S.-style policy debate norms; may degrade outside English, outside standard topic areas, or with sparse evidence
Hierarchical pipeline prone to error propagation, prompt drift, and non-determinism; degenerate failure modes (repetitive arguments, strategic collapse) require stage re-runs
Not stress-tested against adversarial opponents or poisoned evidence (prompt injection via retrieved cards)
Full rounds cost $1–3 USD in LLM tokens (up to $50 with TTS + talking-head video); depends on proprietary APIs

Relevance to Agentic AI / LLM Agents¶

DeepDebater is a concrete demonstration of hierarchical multi-agent orchestration tackling a long-horizon, multi-turn, strategically adversarial task that requires planning, retrieval-augmented generation, self-correction, and sequential conditioning across many pipeline stages — all core problems in agentic AI research. Its role-decomposed workflow pattern (specialized agents per debate "argument type," iterative critic loops, Pydantic-enforced structured outputs) is directly transferable to other complex creative or analytical pipelines. The system also surfaces important failure modes for agentic pipelines: error propagation across dependent stages, non-determinism, and the gap between "winning the task metric" (winning rounds) and truthfulness — a microcosm of objective misalignment at the agent level. The dual-use discussion on superpersuasion at scale is particularly relevant for researchers thinking about societal risks of capable LLM agents operating autonomously in open-ended, high-stakes domains.