Skip to content

The Surface You Test Is Not the Surface That Breaks

🕒 Published (v1): 2026-05-28 18:26 UTC · Source: Arxiv · Venue: EMNLP · link

Why this paper was selected

Prompt injection attack surface is larger than single-vector evaluation suggests

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Tool-augmented LLM agents expose two distinct prompt-injection surfaces — tool outputs (data surface) and tool descriptions (schema surface) — yet current evaluations only test the former. Across 13 LLMs and 6,830 attack attempts with a byte-identical payload, vulnerability fully inverts across models: GPT-4.1 is 96% vulnerable on data but 4% on schema, while Gemini-3-Flash shows the mirror (20% data, 98% schema). Surface preference is a stable model×surface interaction, and adaptive attackers who exploit this gap outperform the best fixed-surface baseline by +9.1 percentage points.

Problem

Prompt-injection benchmarks and defenses fix the attack channel to tool outputs and report a single ASR per model. This single-surface convention has two blind spots: (1) the tool-description (schema) surface — read by the agent at every turn before any tool call — is equally injectable, and (2) the relative danger of each surface is strongly model-dependent, so averaging over models reports the wrong quantity. Standard prompt-level defenses designed for the data stream leave the schema channel essentially unguarded.

Method

The authors build on AgentDojo (four task suites: banking, slack, travel, workspace) and construct a cross-surface evaluation: for every (user-task, injection-task) pair, the identical byte string (a workflow_dependency template containing a trigger condition, plausible justification, urgency marker, and attacker instruction) is injected either into a tool's return value (data surface) or into the same tool's description field (schema surface). Only the delivery location changes; payload, task, and success predicate are held constant.

Three metrics are defined at the cell level \(c = (m, s)\) for model \(m\) and task suite \(s\): - \(\text{ASR}(c, v) = \frac{1}{|U \times I|} \sum_{u,i} y_{c,v,u,i}\) — standard single-surface attack success rate - \(\text{AAR}(c) = \max\!\bigl(\text{ASR}(c,\text{data}),\, \text{ASR}(c,\text{schema})\bigr)\) — Adaptive Attack Rate, the oracle surface-adaptive attacker - \(\text{SOM}(c) = |\text{ASR}(c,\text{schema}) - \text{ASR}(c,\text{data})|\) — Surface-Optimal Margin

A one-way variance decomposition partitions 6,830 binary attempt outcomes across factors (surface, model, suite, and interactions). Family-prior and K-probe fingerprinting strategies are evaluated to measure how cheaply an attacker can identify the better surface without oracle access.

Key Contributions

  • Cross-surface evaluation framework: byte-identical payloads delivered through data and schema surfaces across 13 LLMs × 4 suites (52 cells); shows the model×surface interaction accounts for 16.7% of attempt-level variance while surface alone accounts for 0%.
  • Adaptive Attack Rate (AAR): formalizes the surface-adaptive attacker as the per-cell maximum; shows AAR = 46.5% vs. 37.5% (always-data) and 31.5% (always-schema), a +9.1 pp lift (bootstrapped 95% CI [+4.4, +14.3], Wilcoxon \(p < 0.001\), \(n = 52\)).
  • Surface fingerprinting: within-family historical data captures 46–56% of oracle gain with zero queries; 5 direct probes capture 73%; 10 probes capture 80%.
  • Defense-side surface asymmetry: standard prompt-level defenses (repeat-user-prompt, spotlighting-with-delimiters) reduce data-surface ASR to 10–18% but leave schema-surface ASR above 54%; only a description-aware tool_filter suppresses both.

Results

  • Variance decomposition (N = 6,830): surface alone 0.0%, model alone 6.5%, suite alone 8.9%, model×surface interaction 16.7%, suite×surface 9.4%, model×injection-task 34.4%.
  • Cell-level extremes: GPT-4.1 on slack = −92 pp gap (96% data, 4% schema); Gemini-3-Flash on slack = +78 pp gap (20% data, 98% schema).
  • Aggregate AAR: 46.5% vs. best fixed-surface (37.5%), +9.1 pp lift; lift is consistent across suites (+9.3 pp banking, +8.5 pp slack, +9.2 pp travel, +5.1 pp workspace).
  • Lift decomposition: removing both schema-preferring frontier models reduces lift to +2.1 pp (universal component); those two models contribute a +7.0 pp outlier component.
  • Defenses: repeat_user_prompt → data ASR 10.6%, schema ASR 54.8%; spotlighting_with_delimiting → data 17.9%, schema 54.1%; tool_filter → data 0.0%, schema 0.8%.
  • Native MCP replication: Gemini-3-Flash 0% data / 100% schema; GPT-4.1 100% data / 50% schema; per-target AAR 87.5%, +15 pp over better fixed surface.

Limitations

  • Evaluation uses a single payload template (workflow_dependency); content-optimized attacks would only exceed reported AAR, so results are a lower bound on attacker capability rather than an upper bound.
  • Defense evaluation is preliminary (three-model panel for Table 3), so defense residuals may not generalize across the full 13-model panel.
  • Only two attack surfaces are tested; other injection surfaces (system-prompt poisoning, memory stores, multi-hop retrieval chains) are outside scope.
  • Leave-one-family-out family-prior collapses below the always-data baseline (30.5% vs. 37.5%), meaning the family signature is non-transferable across families — attacker must have within-family priors or spend probes.
  • AgentDojo task suites cover a specific domain distribution (banking, slack, travel, workspace); generalization to other agentic task domains is unverified.

Relevance to Agentic AI / LLM Agents

This paper directly challenges the evaluation methodology underlying most agentic security work: the implicit assumption that measuring one injection channel characterizes a model's full vulnerability. For practitioners building tool-calling agents (RAG pipelines, MCP servers, coding assistants), the finding that tool descriptions are a live attack surface — one that standard defenses completely miss — requires revisiting threat models at the architecture level, not just the prompt level. The Adaptive Attack Rate metric provides a concrete, drop-in improvement to existing benchmarks like AgentDojo that the community can adopt immediately. The model×surface interaction also has implications for agent red-teaming: a red team that evaluates only on tool outputs will systematically underestimate risk for schema-preferring models like Gemini-3-Flash.