Skip to content

Recursive Knowledge Synthesis for Multi-LLM Systems: Stability Analysis and Tri-Agent Audit Framework

🕒 Published (v1): 2025-12-17 16:42 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper proposes a tri-agent cross-validation framework (Semantic + Analytical + Transparency Audit modules, each a separate LLM session) connected by mandatory human-mediated bridging rather than automated APIs. It formalizes this recursive loop as "Recursive Knowledge Synthesis" (RKS), grounded in Banach fixed-point convergence theory, and evaluates stability across 47 human-supervised trials using hand-scored metrics. The central claim is that a human-in-the-loop multi-LLM architecture can achieve stable, auditable knowledge synthesis without automated orchestration infrastructure.

Problem

Existing multi-agent LLM systems (AutoGPT, debate-based reasoning, LLM-as-a-Judge) use automated API routing that risks uncontrolled feedback loops, emergent coordination artifacts, optimization drift, and poor auditability. No prior work provides a formal stability guarantee (contraction mapping) for heterogeneous, cross-vendor LLM ensembles, nor demonstrates that such systems remain stable under realistic freemium deployment conditions (no version pinning, no API access).

Method

Three commercial LLMs are assigned distinct roles via Session-Level Role Decomposition (SLRD): ChatGPT (Semantic Reasoning, MS), Gemini (Analytical Consistency, MA), and Copilot (Transparency Audit, MT). All inter-session communication is manually performed by a single human Supervisor under Human-Bridge Orchestration (HBO) — no automated API calls, no agent-to-agent messaging. The Supervisor copies outputs between isolated browser sessions, reviews each transfer, and injects corrective prompts.

The state transition is modeled as VOp = MT ∘ MA ∘ MS, formalized as a contraction mapping (Banach Fixed-Point Theorem) where MT is the contractive operator that enforces TS ≥ 0.7 compliance, driving the knowledge state vector to a fixed point. Four metrics are scored by the human Supervisor against a standardized rubric: Reflex Reliability Score (RRS = 0.3·TS + 0.4·DDR + 0.3·CSR), Transparency Score (TS), Deviation Detection Rate (DDR), and Correction Success Rate (CSR). Pre-seeded logical/ethical inconsistencies from a Predefined Contradiction Set measure DDR.

Key Contributions

  • Tri-agent, cross-vendor (OpenAI / Google / Microsoft) validation architecture with formally defined role decomposition
  • Formal RKS model: composite validation operator as a contraction mapping with convergence guarantee via Banach Fixed-Point Theorem
  • Human-Bridge Orchestration (HBO) protocol specifying no automated API routing, no agent-to-agent messaging, full human auditability at every transfer
  • Session-Level Role Decomposition (SLRD) as a workflow safety primitive enabling drift detection between isolated conversational contexts
  • Empirical stability evaluation under freemium, non-API-controlled public deployment conditions (47 trials, October 2025)

Results

  • Mean RRS = 0.78 ± 0.06 across 47 trials
  • TS ≥ 0.8 (high compliance): ~68% of trials; 0.7 ≤ TS < 0.8: ~24%; TS < 0.7: ~8% (total ≥ 0.7 threshold: ~92%)
  • Convergence rate: ~89% of trials reached a stable knowledge state
  • Mean convergence time: 12.3 iterations ± 3.7 SD
  • No external baselines compared; all claims are relative to the system's own internal metrics

Limitations

  • All metrics (RRS, TS, DDR, CSR) are hand-scored by a single Supervisor using a qualitative rubric — no inter-rater reliability reported; no automated or API-derived measurements
  • No bit-level reproducibility: freemium LLMs update continuously with no version pinning; study is explicitly a snapshot feasibility study (October 2025)
  • Single annotator across 47 trials — small scale, no statistical independence
  • "Multi-agent" architecture is operationally a human manually copy-pasting between browser tabs; the formal contraction mapping claim is applied to an informal, qualitative process
  • No comparison against automated baselines; the mathematical framework (Banach theorem) is asserted but not empirically validated beyond the hand-scored convergence rate
  • Not scalable: HBO explicitly prohibits automation, making N-agent extension impractical without redesign

Relevance to Harnesses / Meta-Harnesses

This paper defines a deliberate anti-harness design pattern: it replaces an automated orchestration harness with a human operator as the sole coordination layer, arguing that full automation introduces safety and auditability risks. For harness researchers, this is a meaningful negative reference point — it articulates precisely what a harness abstracts away (state transfer, routing, convergence checking) and frames those abstractions as optional rather than necessary. The Session-Level Role Decomposition and Human-Bridge Orchestration protocols are directly translatable to harness design primitives: session isolation maps to sandboxed agent contexts, and the Supervisor's transfer-review step maps to a harness-level validation gate between pipeline stages. However, the paper's operationalization is too informal (single human scorer, no API) to serve as an engineering blueprint, and the formal stability analysis would need to be validated against an actual automated harness to be actionable.