Skip to content

FinDebate: Multi-Agent Collaborative Intelligence for Financial Analysis

🕒 Published (v1): 2025-09-22 06:56 UTC · Source: Arxiv · Venue: EMNLP 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

FinDebate is a multi-agent framework for institutional-grade financial analysis that pipelines five domain-specialized analyst agents (earnings, market, sentiment, valuation, risk) through a domain-specific RAG module and then a constrained three-agent debate pass. The "safe debate protocol" enforces stance preservation—agents can strengthen evidence and risk framing but cannot flip directional recommendations—yielding calibrated, auditable reports. Evaluated on the Earnings2Insights shared task, FinDebate raises LLM-judge scores by ~0.6 points (~20%) over zero-shot across five backbone models.

Problem

Existing financial AI pipelines suffer from three compounding failures: (1) LLM overconfidence and miscalibrated stance drift during multi-round revision; (2) RAG integration producing incoherent multi-section narratives where evidence coverage is uneven across analytical dimensions; (3) multi-agent debate frameworks that introduce topic drift and directional reversals in long-form, chaptered reports, making outputs unsuitable for regulated investment contexts.

Method

FinDebate is a three-stage pipeline:

  1. Domain-specific RAG: Earnings call transcripts are segmented with a recursive, semantic-boundary-respecting chunker (paragraph → sentence → token priority), encoded with FinLang (BGE fine-tuned on financial corpora), and stored in ChromaDB. Retrieval covers four query dimensions: general financial performance, specialized financial metrics (NIM, ROA, NPAs), market sentiment & risk, and multi-query integration.

  2. Multi-agent draft generation: Five role-specialized agents run in parallel, each with a two-level prompt (system: professional identity + credentials; user: analytical framework + output spec + RAG context). A Report Synthesis agent consolidates outputs into a single chaptered draft.

  3. Safe Collaborative Debate (Algorithm 1): A single-round, three-agent refinement pass over the draft. The Trust Agent strengthens supporting evidence without altering directional tone or time-horizon recommendations. The Skeptic Agent adds risk factors and hedge strategies. The Leader Agent synthesizes both into the final report. A safety check at entry and exit enforces structural integrity; if core recommendations are compromised, the original report \(R_0\) is returned unchanged. Only one debate round is used (two-round ablation confirmed marginal returns with added drift risk).

Key Contributions

  • A safety-constrained debate protocol that enforces stance immutability (\(\text{direction}(R^*) = \text{direction}(R_0)\)) while expanding evidence coverage and risk articulation.
  • A domain-specific RAG stack combining semantic chunking, FinLang embeddings, and four-dimensional multi-level retrieval tailored to financial constructs.
  • A role-specialized five-agent analysis module covering the five institutional investment dimensions.
  • Model-agnostic design validated across GPT-4o, Gemini 2.5 Flash, Llama 4 Maverick, DeepSeek-R1, and Claude Sonnet 4.
  • Human evaluation showing superior investment decision accuracy and report quality (clarity, logic, persuasiveness, usefulness) over competing teams on Earnings2Insights.

Results

  • LLM-judge (4-point scale): FinDebate scores 3.39–3.64 vs. 2.77–3.03 zero-shot baseline; uniform ~+0.61 gain across all five backbone models (\(p < 0.001\), paired \(t\)-test); represents ~20.4% average improvement.
  • Ablation: Multi-agent without debate scores 3.10–3.45; adding the debate mechanism adds ~0.19 points on top.
  • Human evaluation (financial decision accuracy): FinDebate outperforms other Earnings2Insights teams on day/week/month average accuracy (exact figures shown in Figure 2 as bar chart comparisons; FinDebate leads in all time horizons).
  • Human evaluation (report quality Likert): FinDebate scores ~5.6–6.0 vs. competing teams' ~5.2–5.4 across clarity, logic, persuasiveness, readability, and usefulness (Figure 3).
  • Evaluated on 10 ECTSum + 5 professional analyst report samples from Earnings2Insights.

Limitations

  • Single-round debate is a design choice justified empirically but not thoroughly ablated at scale; the optimal round count may vary by domain or document complexity.
  • Evaluation set is small (15 reports total for human eval), limiting statistical generalizability.
  • The safe debate protocol is explicitly inapplicable to report generation from scratch or to texts lacking directional investment conclusions—restricting scope to refinement-only use cases.
  • No latency or cost analysis; parallelizing five agents plus a three-agent debate round at inference time may be prohibitive in low-latency production settings.
  • Domain coverage is limited to earnings call transcripts; extension to other financial document types (10-K, macroeconomic reports) is deferred to future work.

Relevance to Harnesses / Meta-Harnesses

FinDebate is a concrete domain-specific multi-agent harness that operationalizes several patterns central to harness design: parallel specialist fan-out feeding a synthesis agent, a constrained refinement pass with explicit safety invariants, and a fallback mechanism (\(R_0\) returned if integrity is violated). The "safe debate" protocol is particularly relevant as a model for how harnesses can enforce behavioral contracts over sub-agents—preventing downstream agents from overriding upstream conclusions—analogous to harness-level guardrails in general orchestration frameworks. The single-round cap and stance-lock constraint demonstrate deliberate harness-level control flow that trades coverage for stability, a tradeoff generalizable beyond finance.