Echoing: Identity Failures when LLM Agents Talk to Each Other

🕒 Published (v1): 2025-11-12 20:17 UTC · Source: Arxiv · Venue: ICLR · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLM agents in autonomous agent-agent (AxA) conversations exhibit a failure called "echoing," where an agent abandons its assigned role and mirrors its conversational partner. This failure occurs in 5–70% of interactions depending on model and domain, persists in advanced reasoning models, and is masked by standard task-completion metrics. A protocol-level mitigation using structured responses reduces echoing to ~9% but cannot eliminate it.

Problem

Existing LLM agent evaluation frameworks are designed for single-agent or human-supervised settings. When two LLM agents interact autonomously with private, misaligned objectives, a new class of emergent behavioral failure arises: agents may drift from their assigned identities. No prior work systematically characterizes or measures this AxA-specific failure, and standard task-completion metrics cannot detect it (93% of echoing conversations still complete successfully).

Method

The authors formalize AxA interactions as a partially observable stochastic game, where each agent \(A_i\) is characterized by identity \(I_i\), objective \(O_i\), tools \(T_i\), utility \(U_i\), and LLM policy \(\pi_i\). Echoing is defined as the event where agent \(A_i\) generates responses whose language or decisions align with the partner's identity \(I_j\) rather than its own \(I_i\).

They run 66 AxA configurations across 4 domains (car sales, hotel booking, supply chain, medical consultation), 22 customer-agent model variants (GPT-4o, GPT-4.1, o3, GPT-5, Gemini-2.5-Flash/Pro, Claude Sonnet-4, Llama-3.1-8B/70B) paired against 3 seller configurations, and 3 prompt formulations (minimal, behavioral, identity boundary). Echoing is detected using an LLM-as-judge (GPT-4o with structured output) that analyzes full conversation histories for identity inconsistency; validated against 150 human-annotated samples (91.1% agreement). Mitigation is tested via a Pydantic-style structured response format that forces agents to declare their role explicitly in every turn.

Key Contributions

• Formal definition of AxA interactions as a partially observable stochastic game and introduction of "echoing" as a quantifiable identity-consistency failure metric (EchoEvalLM).
• Large-scale empirical study (2,500+ conversations, 250,000+ LLM inferences) establishing echoing prevalence (5–70%) across all major LLM providers.
• Demonstration that increased test-time reasoning effort does not eliminate echoing (32.7/32.8/32.9% for low/medium/high reasoning vs. 37.7% for non-reasoning).
• Temporal analysis showing echoing onset typically occurs at turn 7–8, with likelihood increasing with conversation length.
• Protocol-level mitigation via structured responses reducing echoing to <10% in GPT and Sonnet model families.

Results

• Echoing prevalence: 5–70% across models and transactional domains; Gemini-2.5-Flash worst (64–73% across domains), GPT-5 best (2–10%), Llama-3.1-70B surprisingly low (9.1%).
• Reasoning vs. non-reasoning: Non-reasoning average 37.7%; reasoning (all effort levels) average 32.6–32.9% — marginal and not significant improvement.
• Prompt design: Identity boundary prompts (explicit anti-echoing instructions) reduce but do not eliminate echoing; Gemini-2.5-Flash holds 64–73% across all three prompt variants.
• Domain sensitivity: GPT-4o shows 58% (car sales), 25% (hotel booking), 17% (supply chain) — 41% swing. Advisory/medical domain: substantially lower rates, with most models showing <5% except Llama-3.1-8B (30%) and Gemini-2.5 (15–18%).
• Temporal dynamics: Echoing onset averages turn 7.6 (median 8.0); echoing conversations are on average 0.9 turns longer than non-echoing ones (9.6 vs. 8.7).
• Structured response mitigation: Reduces echoing to below 10% in GPT and Sonnet variants across all 3 transactional domains (24 configurations tested).
• Task completion masking: 93% of conversations with echoing still complete the transaction, hiding the failure from standard metrics.

Limitations

• Only customer-seller and one advisory scenario tested; multi-party, collaborative, or long-horizon AxA contexts not studied.
• Seller agent configurations limited to 3 variants vs. 22 customer variants — echoing asymmetry not fully characterized from the seller side.
• Echoing metric relies on LLM-as-judge; cross-model judge consistency is 79.6–88.9%, and no alternative detection strategies explored.
• Gemini-2.5 models excluded from structured response mitigation experiments due to tool + structured output compatibility failures.
• No mechanistic analysis of open-weight model activations to understand why Llama models resist echoing.
• Protocol-level identity refresh (role reinstatement every N turns) produced disordered conversation flow and was not pursued.

Relevance to Agentic AI / LLM Agents

Echoing is a fundamental reliability challenge for any production multi-agent system where LLM agents interact autonomously without human oversight — directly relevant to emerging agent-to-agent protocols (Google A2A, IBM BeeAI, Cisco Agntcy). The finding that task-completion metrics conceal identity drift means current evaluation standards are insufficient for certifying AxA reliability, pushing the field toward behavioral consistency metrics. The persistence of echoing across reasoning models and explicit anti-drift prompts implies that safe multi-agent deployments will require training-time interventions or architectural mitigations, not just better prompting. For researchers designing LLM agent pipelines, this work provides both a concrete threat model and a near-term mitigation (structured role-declaring responses) applicable today.