Skip to content

AI4Reading: Chinese Audiobook Interpretation System Based on Multi-Agent Collaboration

🕒 Published (v1): 2025-12-29 08:41 UTC · Source: Arxiv · Venue: ACL 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AI4Reading is a multi-agent LLM-based system that automatically generates podcast-style interpretive audiobooks in Chinese by orchestrating 11 specialized agents across topic analysis, case enrichment, oral rewriting, and reconstruction stages. Built on MetaGPT + DeepSeek-V3, it produces interpretive scripts (not just summaries) and converts them to speech via Fish-Speech TTS. Human evaluation against the FanDeng professional platform shows the system's scripts are simpler and more accurate, though TTS naturalness lags behind human narrators.

Problem

Manual creation of interpretive audiobooks (as distinct from abridged or summarized versions) requires skilled editors, narrators, and sometimes authors working in concert—making it expensive, slow, and non-scalable. Single-LLM approaches with CoT or RAG collapse to summarization rather than genuine interpretation: they lack real-world case elaboration, narrative expansion, and the content volume required for full chapter coverage.

Method

A pipeline of 11 specialized agents built on MetaGPT (DeepSeek-V3 backbone, temp=1.3, 0-shot, max 8192 tokens) executes four sequential stages per chapter:

  1. Topics & Cases Identification (TCI): Topic Analyst (TA) extracts ≤3 core topic-case pairs; Proofreader PR-1 validates and rejects invalid pairs (triggering TA retry); Case Analyst CA-1 enriches cases with background detail.
  2. Preliminary Interpretation (PI): CA-2 adds personal anecdotes/real-life perspectives; CA-3 constructs causal arguments per topic; Editor ED-1 synthesizes a preliminary draft; Proofreader PR-2 evaluates completeness and logicality with iterative revision (max Imax=3 rounds).
  3. Oral Rewriting (OR): Narrator NR simplifies syntax and injects colloquial markers; PR-3 evaluates naturalness and fluency, triggering iterative NR refinement.
  4. Reconstruction & Revision (RR): Editor ED-2 incrementally merges per-topic oral scripts into a full manuscript; PR-4 evaluates global coherence and fluency with iterative revision.

The final manuscript is converted to audio via Fish-Speech TTS with chapter-transition sound effects inserted.

Key Contributions

  • First system for automated interpretive audiobook generation (not summarization/simplification) using LLMs + TTS, supporting Chinese books and Chinese interpretations of English books.
  • 11-agent collaborative framework decomposing interpretation into topic identification, case analysis, oral rewriting, and manuscript reconstruction with iterative proofreader feedback loops.
  • Empirical justification for multi-agent over single-LLM: single-pass CoT defaults to summarization and produces insufficient content volume.
  • Public codebase and human evaluation methodology comparing AI output against expert professional platform (FanDeng).

Results

Human evaluation (5 valid annotators, 1–5 Likert scale, 10 excerpts from 5 books) vs. FanDeng (expert human narrators):

  • Textual content — AI4Reading wins on all four dimensions:
  • Simplicity: 4.6 (ours) vs. 4.4 (FD)
  • Completeness: 4.0 (ours) vs. 3.8 (FD)
  • Accuracy: 4.3 (ours) vs. 4.2 (FD)
  • Coherence: 4.4 (ours) vs. 4.1 (FD)
  • Audio quality — FanDeng wins on all three dimensions:
  • Naturalness: 4.1 (ours) vs. 4.9 (FD)
  • Concentration: 3.4 (ours) vs. 4.2 (FD)
  • Comprehension: 3.1 (ours) vs. 3.3 (FD)
  • Average generated audio duration: 4 min 59 s (ours) vs. 4 min 33 s (FD)

Limitations

  • Very small evaluation sample: 5 valid annotators, 10 excerpts from 5 books in 2 genres (personal growth, business); generalizability is unestablished.
  • Domain restricted to psychology, personal growth, and business management; literature/novels not supported.
  • TTS quality (naturalness, listener concentration) clearly inferior to professional human narrators.
  • No automated metrics; purely manual evaluation limits reproducibility and scalability of assessment.
  • Evaluation excludes comparison against simpler baselines (e.g., single-LLM with longer prompts, RAG), making the marginal gain from the 11-agent architecture hard to isolate.

Relevance to Agentic AI / LLM Agents

This paper is a concrete case study of role-specialized multi-agent pipelines for long-form creative generation, demonstrating the practical design principle that decomposing a complex task into narrow-scope agents with iterative proofreader feedback loops outperforms monolithic CoT approaches. The iterative revision mechanism (editor→proofreader→editor) operationalizes self-correction within a multi-agent graph, a pattern directly relevant to agentic reliability research. The explicit failure analysis of single-LLM approaches (summarization collapse, content thinning) provides empirical grounding for why agentic decomposition is necessary for tasks requiring content expansion rather than distillation. It also illustrates MetaGPT as an orchestration substrate for domain-specific agent systems beyond software engineering.