EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning

🕒 Published (v1): 2026-06-13 06:05 UTC · Source: Arxiv · Venue: INTERSPEECH 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EChO-Agent is a modular, tool-augmented agent framework that decomposes complex audio question answering into a four-stage pipeline: Tool → Evidence → Reason → Verify. It addresses the failure of Large Audio Language Models (LALMs) to produce grounded, checkable reasoning chains by interposing an LLM-based evidence integration stage that distills heterogeneous tool outputs into compact, question-conditioned evidence before passing them to the LALM backbone. On the MMAR benchmark it achieves 71.0% accuracy and a 63.0 rubric score, ranking 5th in the Agent Track.

Problem

LALMs lack question-conditioned perception, cannot revisit audio after initial encoding, and produce reasoning chains that are weakly grounded or unfaithful to audio evidence. Existing tool-augmented approaches (AuTAgent, AudioRouter, CoFi-Agent, AudioRAG) focus on what information is acquired but pass raw tool outputs directly to the reasoner, introducing distracting context and offering no mechanism to synthesize, filter, or verify evidence–answer consistency. The Interspeech 2026 Audio Reasoning Challenge explicitly evaluates reasoning-path faithfulness via instance-level rubrics, not just answer correctness—a gap existing systems do not close.

Method

The pipeline comprises four sequentially orchestrated stages:

1. Tool-Augmented Observation: A question-type-conditioned static dispatch strategy (not stochastic) selects a predefined tool combination from four specialist tools—YAMNet (audio event detection), Whisper (ASR), SpeechBrain wav2vec2 (speech emotion recognition), and Essentia (music information retrieval). Each tool receives a question-rephrased directive \(p_j\); failed calls are retried up to 2 times and logged as [UNAVAILABLE] to suppress hallucination. Output: observation set \(O\).

2. LLM-Based Evidence Integration: DeepSeek-V3 distills \(O\) into a compact evidence set \(E\) via three structured operations: relevance filtering (discard tangential content), cross-observation synthesis (resolve conflicts by confidence/specificity), and evidence structuring (order facts in decision-critical sequence). This is the "knowledge bridge" stage.

3. Evidence-Conditioned Audio Reasoning: Qwen-3-Omni-Instruct receives the raw audio \(a\), question \(q\), candidate options, and structured evidence \(E\) jointly. The reasoning is run twice with different temperatures and reordered evidence to produce two candidates \(\hat{y}^{(1)}\) and \(\hat{y}^{(2)}\). On failure, diagnostic feedback \(f^{(r-1)}\) is injected into the next iteration prompt: $\(\hat{y}^{(r)} = M_A\!\left(\pi_{\text{ans}},\, a,\, q,\, \{c_i\},\, E,\, f^{(r-1)}\right)\)$

4. Verification and Dual-Pass Arbitration: The LLM orchestrator \(M_L\) performs (a) format compliance repair, (b) reasoning–answer consistency checking, and (c) dual-pass arbitration: $\(y^* = M_L\!\left(\pi_{\text{ver}},\, q,\, E,\, \hat{y}^{(1)},\, \hat{y}^{(2)}\right)\)$ Agreement → shared answer adopted; disagreement → verifier selects the candidate with stronger evidence alignment.

Key Contributions

• A four-stage Tool → Evidence → Reason → Verify pipeline that separates perception, evidence distillation, reasoning, and verification into distinct, auditable modules.
• A structured evidence integration stage (using DeepSeek-V3) that filters, synthesizes, and orders multi-tool outputs into a compact, question-conditioned evidence chain before the LALM ever sees it.
• A question-type-conditioned static tool dispatch strategy that eliminates variance from stochastic tool selection.
• A dual-pass arbitration verification mechanism providing implicit self-consistency checking without expensive majority voting.
• Ablation evidence establishing that unfiltered tool outputs hurt reasoning (w/o Evidence Integration drops below the tool-free LALM baseline).

Results

• Full pipeline: 71.0% accuracy, 63.0 rubric score on MMAR; ranked 5th on the MMAR Agent Track.
• vs. Qwen-3-Omni-Instruct baseline: +2.3 accuracy points (68.7 → 71.0), +4.3 rubric points (58.7 → 63.0).
• Best average accuracy among all compared LALMs in Table 1 (including Gemini 2.0 Flash at 65.6% avg and Qwen-3-Omni-thinking at 69.0% avg).
• Ablations on MMAR:

Variant	Accuracy	Rubrics
Full Pipeline	71.0	63.0
w/o Verification	69.1	61.5
w/o Observation	69.2	60.2
w/o Evidence Integration	65.4	56.9
Qwen3-Omni-Instruct (baseline)	68.7	58.7

• Removing evidence integration causes the largest drop (−5.6% accuracy, −6.1 rubrics), falling below the tool-free baseline—confirming that raw tool outputs without filtering are harmful.

Limitations

• Perception tool granularity: YAMNet produces coarse frame-level event labels, capping fine-grained sound-centric reasoning; this is explicitly identified as a bottleneck.
• Unresolved tool uncertainty and cross-tool conflicts: Noted as future work; the current pipeline resolves conflicts by confidence heuristics but lacks principled uncertainty modeling.
• No fine-grained temporal analysis: The pipeline lacks mechanisms for precise temporal localization within audio clips.
• Static tool dispatch: The question-type-conditioned dispatch eliminates variance but also eliminates adaptability; it cannot dynamically reselect tools based on intermediate observations.
• Modest absolute gains: +2.3 accuracy points over the backbone LALM is real but small; the rubric gain (+4.3) is more meaningful as a faithfulness metric.

Relevance to Harnesses / Meta-Harnesses

EChO-Agent is a concrete instantiation of a domain-specific orchestration harness: it wraps a LALM backbone with a fixed four-stage meta-pipeline that plans tool use, executes specialist tools, integrates their outputs, and verifies the final answer—exactly the structure a meta-harness provides over component agents. The key design insight transferable to general harness engineering is the evidence integration layer as a mandatory intermediary: raw tool outputs fed directly to a reasoning model degrade performance, and a structured distillation step (filtering, synthesis, ordering) is required to make tool-augmentation net-positive. The dual-pass arbitration with diagnostic feedback injection is a lightweight self-repair loop that harness designers can adopt without majority voting overhead. The ablation showing that naive tool augmentation without evidence structuring is worse than no tools at all is a strong empirical warning for any harness that pipes raw sub-agent outputs directly into a final reasoning step.