Towards Responsible and Explainable AI Agents with Consensus-Driven Reasoning¶

🕒 Published (v1): 2025-12-25 14:49 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper proposes a multi-agent architecture that runs a consortium of heterogeneous LLMs/VLMs in parallel isolation and then routes all candidate outputs to a dedicated reasoning-layer governance agent for structured consolidation, conflict resolution, and policy enforcement. The architecture treats explainability (via cross-model disagreement exposure) and responsibility (via centralized reasoning-layer control) as first-class architectural properties rather than afterthoughts. It is evaluated across five real-world agentic workflows spanning content generation, biomedical imaging, psychiatric diagnosis, and RF signal classification.

Problem¶

Single-model agentic pipelines collapse uncertainty into one output with no independent perspective, making them prone to hallucinations, reasoning drift, and untraceable decisions; they also lack explicit governance mechanisms for policy enforcement, bias mitigation, and auditability. Existing ensemble methods rely on voting or heuristic aggregation without a dedicated agent capable of structured meta-reasoning or safety-constrained synthesis.

Method¶

A consortium of heterogeneous LLMs and VLMs (e.g., GPT, Gemini, Claude, Llama, Pixtral, Qwen) each receives an identical canonical prompt and shared input context, executing fully in parallel with no inter-agent visibility. Their outputs are preserved verbatim as first-class artifacts. A reasoning-layer governance agent (implemented with a reasoning-focused LLM, e.g., OpenAI GPT-o series) then performs structured meta-evaluation: cross-model comparison, conflict detection, factual alignment, redundancy removal, relevance filtering, and explicit identification of speculative or unsupported claims. It synthesizes a single consolidated output grounded in multi-model agreement, with confidence explicitly downgraded where models diverge. All intermediate outputs are retained for audit. The same coordination pattern is applied uniformly across all five evaluated domains.

Key Contributions¶

Formal architectural separation of XAI (multi-model consensus exposes uncertainty/alternatives) from RAI (centralized reasoning layer enforces safety, policy, and auditability)
A generalizable production-grade agent architecture: LLM/VLM consortium + reasoning-layer governance agent applicable across modalities (text, image, multimodal)
Demonstration of the architecture across five diverse, high-stakes agentic workflows: news podcast generation, neuromuscular H-reflex analysis, tooth-level gingivitis detection, psychiatric DSM-5 diagnosis, and RF signal classification in 5G security

Results¶

No quantitative accuracy metrics or ablation numbers are reported; evaluation is qualitative and behavioral: - Podcast generation: consensus-driven approach "substantially reduces hallucinated statements" and "improves factual consistency" vs. single-model baselines; clear content provenance is maintained - Neuromuscular reflex: "improves diagnostic robustness, reduces interpretation bias" vs. single-model baselines - Dental gingivitis: reasoning agent amplifies high-confidence tooth-level findings, flags borderline predictions as lower-confidence, and recommends secondary review where models diverge substantially - Psychiatric diagnosis: "improves diagnostic consistency, reduces idiosyncratic model behavior" vs. single-model pipelines; DSM-5 alignment verified - RF signal classification: multi-model Unknown consensus yields high-confidence anomaly labels; conflicting classifications are flagged for cautious handling rather than overconfident assignment

Limitations¶

No quantitative benchmarks, ablation studies, or statistical comparisons against baselines are provided; all claims are qualitative
Latency and cost of running N parallel LLM/VLM agents plus a reasoning agent are not analyzed
The reasoning agent itself is a single model (GPT-o series) and introduces its own single-point-of-failure risk not addressed architecturally
Fine-tuned consortium models for biomedical use cases are described but training details and dataset provenance are not disclosed
Policy and safety constraints are described conceptually but not specified in detail; generalizability of the governance mechanism to new domains is assumed, not demonstrated
No formal adversarial evaluation (e.g., prompt injection resilience, failure-mode stress tests)

Relevance to Harnesses / Meta-Harnesses¶

This paper is a direct instantiation of the meta-harness pattern: a reasoning agent that orchestrates and arbitrates over multiple specialist sub-agents is precisely the architecture a harness-tracking researcher cares about. The paper makes explicit the two roles a meta-harness must play—fan-out orchestration (parallel isolated execution over a shared context) and fan-in governance (structured consolidation with policy enforcement)—and argues these must be architecturally separated. The "reasoning-layer governance agent" is functionally a meta-harness controller: it receives all sub-agent outputs, runs structured meta-evaluation logic, and produces a single auditable result. The consistency of this pattern across five domains with different modalities (text, image, multimodal spectrogram) strengthens the case that consensus+reasoning is a domain-agnostic harness primitive.