Skip to content

A2ASecBench: A Protocol-Aware Security Benchmark for Agent-to-Agent Multi-Agent Systems

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Neil Gong (Duke); first security benchmark for A2A protocol multi-agent systems

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

A2ASecBench is the first security benchmark targeting the Agent-to-Agent (A2A) protocol used in LLM-based multi-agent systems (MAS). The authors define a six-attack threat taxonomy spanning supply-chain manipulations and protocol-logic weaknesses, then evaluate these attacks across travel, healthcare, and finance deployments. Five of six attack types achieve 100% Attack Success Rate (ASR), exposing a systemic absence of protocol-level defenses.

Problem

Existing LLM-MAS security research focuses on prompt-level or topology-level vulnerabilities and lacks coverage of low-level, protocol-specific attack surfaces introduced by the A2A standard (AgentCard discovery, task lifecycle management, artifact exchange). No standardized, reproducible benchmark exists for quantifying security vs. utility trade-offs in A2A-based systems.

Method

The authors construct A2ASecBench around three components:

  1. Threat taxonomy: Six attacks classified into supply-chain manipulations (AgentCard Spoofing, Capability Cloaking) and protocol-logic weaknesses (Cycle Overflow, Half-Open Task Flooding, Agent-Side Request Forgery, Artifact-Triggered Script Injection), each formally modeled (e.g., AS as a multiple-choice selection over \(C^* = \{C^+(a)\} \cup \{C_1^-(a), \ldots, C_k^-(a)\}\) with \(k=10\)).

  2. Dynamic scenario adapter: A mapping \(\text{Adapter}: \mathcal{A} \times \mathcal{S} \rightarrow \mathcal{T}\) instantiated by an LLM that translates abstract attack vectors and domain specifications into concrete executable test cases, enabling portability across heterogeneous agent stacks.

  3. Joint safety–utility evaluation: Adversarial trials are paired with benign task suites; utility degradation is measured as \(\Delta U = U_\text{baseline} - U_\text{cloaked}\), and attack efficacy as Attack Success Rate \(\text{ASR} = \frac{\sum_{i=1}^{N} I_i}{N}\).

The system under test is the official A2A sample MAS (Gemini 2.5 Flash host + 3 remote servers + 3 clients) instantiated in travel, healthcare, and finance domains.

Key Contributions

  • First threat taxonomy and formal threat model for the A2A protocol ecosystem, covering all five lifecycle stages (discovery → initiation → processing → interaction → completion).
  • A2ASecBench: first A2A-specific security benchmark with a domain-agnostic scenario adapter enabling reproducible evaluation across heterogeneous deployments.
  • Empirical system-level evaluation demonstrating that five attacks achieve 100% ASR and AgentCard Spoofing achieves ~0.82–0.83 ASR across all three domains and four frontier models.
  • Practical design principles for stakeholders: progress-aware orchestration, peer protection, and verifiable capability claims.

Results

  • Capability Cloaking, Half-Open Task Flooding, Cycle Overflow, ASRF, ATSI: 100% ASR across all three domains (travel, healthcare, finance).
  • AgentCard Spoofing: average ASR 0.820 (travel), 0.816 (healthcare), 0.828 (finance); consistent across Gemini 2.5 Flash, GPT-4o, Claude 4, DeepSeek-R1; Grok4 performs best but still fails on a fraction of cases.
  • Capability Cloaking utility degradation: benign task performance drops from 0.853→0.682 (travel), 0.872→0.595 (healthcare), 0.962→0.749 (finance).

Limitations

  • Evaluation is restricted to the official A2A sample implementation (Gemini 2.5 Flash); results may not generalize to all A2A-compliant frameworks.
  • No defenses are proposed or tested; the benchmark measures vulnerability without evaluating mitigation effectiveness.
  • AgentCard Spoofing evaluation depends on LLM-generated perturbations (\(k=10\) lookalikes), which may not cover the full adversarial distribution.
  • The scenario adapter relies on an LLM, introducing potential inconsistency or incompleteness in generated test cases.
  • Availability (DoS) attacks such as HOTF and CO are evaluated by proxy (timeout/deadlock) rather than quantitative throughput metrics.

Relevance to Agentic AI / LLM Agents

A2A is rapidly becoming the de facto inter-agent communication protocol (20k GitHub stars in five months), making its security surface a direct concern for anyone building or deploying LLM multi-agent pipelines. This work identifies that trust assumptions baked into A2A's opaque AgentCard model—where agents are accepted as "black boxes"—are exploitable at every lifecycle stage, not just at the prompt level. The benchmark fills a critical gap by providing reproducible, protocol-aware red-teaming tooling that complements existing prompt-injection and topology-level defenses. For researchers building agentic orchestration systems, the six attack vectors (especially Capability Cloaking and ASRF) represent immediate threat classes to audit in production deployments.