Skip to content

MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

MCP tool attack surface benchmark; timely security analysis for agentic deployment

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MSB (MCP Security Bench) is the first end-to-end benchmark for evaluating LLM agent security under the Model Context Protocol (MCP), covering attacks across all three pipeline stages: task planning, tool invocation, and response handling. It defines a taxonomy of 12 attack types, 2,000 attack instances across 405 tools and 10 domains, and a new composite metric—Net Resilient Performance (NRP)—that quantifies the security/utility trade-off. A key finding is that stronger models are paradoxically more vulnerable due to their superior tool-following capabilities.

Problem

Existing agent security benchmarks (AgentDojo, InjecAgent, ASB) operate within the function-calling paradigm and cannot capture MCP-specific attack surfaces. MCP's decentralized, server-hosted tool architecture introduces novel vectors—tool name metadata, natural-language descriptions, out-of-scope parameters, poisoned responses, and retrieval-injected databases—that prior benchmarks systematically miss. No standardized evaluation harness existed for dynamic, real-tool MCP attacks spanning the full agentic pipeline.

Method

MSB formalizes MCP tool-augmented agent behavior via the objective: $\(\mathbb{E}_{q \sim \pi_q}\left[\mathbf{1}\!\left(\text{Agent}(\text{LLM}(p_\text{sys} \oplus T, q, O), T, D) = a\right)\right]\)$ and attacker goals as maximizing \(\mathbb{E}_{q \sim \pi_q}[\mathbf{1}(\text{Agent}(q, \theta^m) = a^m)]\) where \(\theta^m\) encompasses adversarial modifications to the system prompt, tool list, observations, and knowledge database.

Attack taxonomy (12 types across 4 categories): - Tool Signature Attacks (planning stage): Name Collision (NC) appends a suffix \(s\) to \(\tau_n^t\) to hijack tool selection; Preference Manipulation (PM) prepends promotional text to \(\tau_d^t\); Prompt Injection (PI) appends \(x^m\) to \(\tau_d^t\). - Tool Parameter Attack (invocation stage): Out-of-Scope Parameter (OP) sets \(\tau_p^m = i^m\) to induce information leakage. - Tool Response Attacks (response handling): User Impersonation (UI) prepends a fake user turn \(u\) carrying \(x^m\); False Error (FE) fabricates error messages requiring compliance with \(x^m\); Tool Transfer (TT) uses a relay tool to redirect the agent to an endpoint attack tool. - Retrieval Injection (RI): poisons the external database \(D^p \ni x^m\) so benign retrieval tools deliver the malicious instruction. - Mixed Attacks: combinations (e.g., PM-UI, PI-FE) targeting multiple stages simultaneously.

The evaluation harness executes real MCP tools (both benign and malicious), not simulations, with attacks assessed by inspecting actual workspace state (file modifications, process kills, etc.). Metrics: ASR (Eq. 8), PUA (Eq. 9), and NRP \(= \text{PUA} \cdot (1 - \text{ASR})\) (Eq. 10), computed entirely in adversarial settings to account for attack-absent scenarios being structurally different.

Key Contributions

  • Full taxonomy of 12 MCP-specific attack types spanning all three pipeline stages (planning, invocation, response handling).
  • Dynamic evaluation harness executing live MCP tool calls (304 benign + 405 attack tools, 65 tasks, 10 scenarios, 2,000 instances).
  • New metric NRP that jointly captures utility and robustness without conflating benign and adversarial performance.
  • Large-scale evaluation of 10 LLM backbones revealing peak ASR of 75.83% and the counterintuitive finding that stronger models are more susceptible.

Results

  • Peak ASR of 75.83% across 10 evaluated LLM backbones.
  • Stronger models (higher general capability) exhibit higher ASR due to superior instruction-following and tool-use ability—the very properties that make them useful also make them exploitable.
  • MSB identifies attack success across all three MCP pipeline stages, with tool signature attacks, response attacks, and retrieval injection each achieving meaningful success rates.
  • Mixed attacks (e.g., PM-UI, PM-FE, TT-OP) combining multiple vectors amplify impact over single-stage attacks.
  • (Specific per-model NRP/PUA/ASR breakdowns are referenced in appendix tables but not fully reproduced in the provided text excerpt.)

Limitations

  • Evaluation of full per-model per-attack-type breakdown (Tab. 3–9) is in appendices not included in the excerpt, limiting external verifiability from this text alone.
  • Attack tasks are limited to 6 categories (e.g., process kill, file modification); coverage of more diverse real-world harm objectives is not evaluated.
  • Defense mechanisms are not studied—MSB establishes a baseline but does not evaluate mitigations or hardened agents.
  • The benchmark uses Smithery.ai's most popular tools; coverage of long-tail or domain-specific MCP deployments is not characterized.
  • Attacker model assumes white-box access to external resources and ability to deploy MCP servers on third-party platforms; insider-only threat models are not differentiated.

Relevance to Agentic AI / LLM Agents

MCP has rapidly become foundational infrastructure for LLM agent tool use, making its security posture directly relevant to anyone deploying or researching tool-augmented agents. MSB reveals a fundamental tension in agentic system design: the properties that make agents capable (faithful instruction following, strong tool invocation) directly enlarge their attack surface under adversarial tool metadata. The benchmark operationalizes the full agentic loop—planning → tool call → response → retrieval—as an attack surface, which is a more realistic threat model than prior single-stage injection benchmarks. For researchers working on agent safety, tool orchestration, or MCP-based multi-agent systems, MSB provides a concrete evaluation harness and a sobering empirical baseline showing that current frontier models offer weak resistance to straightforward protocol-level adversarial manipulation.