Skip to content

Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoring

🕒 Published (v1): 2026-07-02 12:59 UTC · Source: Arxiv · link

Why this paper was selected

Behavioral side-channel detects guardrail activation without model internals access

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

In production AI systems, defenders can configure guardrail blocks to be indistinguishable from native LLM refusals, yet the evasion strategies for each differ substantially. This paper introduces the first black-box guardrail reconnaissance methodology that uses only 40 probing prompts and HTTP/lexical/timing signals to detect guardrail presence with 100% accuracy and classify future blocks vs. LLM rejections at 98% average F1.

Problem

Existing adversarial attack research assumes the attacker knows whether a refusal was caused by a guardrail middleware layer or by the LLM's own safety alignment—a white-box or grey-box assumption that does not hold in production. Production systems can mask guardrail blocks as natural LLM responses (200 status code, templated refusal text, no distinctive headers), leaving adversaries—and legitimate red-teamers—unable to select the correct bypass strategy or measure attack progress accurately.

Method

A four-stage pipeline operating under strict black-box access (API endpoint only, zero system knowledge):

  1. Response Gathering: Send a benign prompt-set (\(|P_B|=10\)) and up to three category-specific malicious prompt-sets (\(|P_M^{(c)}|=10\) each, covering jailbreak, prompt injection, toxicity) to the target.
  2. Feature Extraction: Collect three feature groups from raw HTTP responses:
  3. HTTP: status codes, response body schema, security-middleware headers.
  4. Lexical: LLM-judge classification of explicit block language; per-response templating prevalence (canned block strings repeat; organic LLM outputs do not).
  5. Timing: elapsed wall-clock time and time-per-token (TPT), estimated as \(\text{TPT} = \text{elapsed}/\max(1, \lfloor\text{chars}/4\rfloor)\); a guardrail that skips LLM generation inflates TPT and reduces elapsed time.
  6. Behavioral Monitoring: Apply Kolmogorov–Smirnov (KS) tests to continuous features and Fisher's Exact (FE) tests to categorical features, comparing malicious vs. benign baselines. All \(p\)-values are corrected via Benjamini–Hochberg FDR; \(q < 0.05/0.01/0.001\) denotes weak/moderate/high confidence of guardrail activation.
  7. Guardrail Characteristic Identification: Per-category signal strengths identify which content types the guardrail covers; fired signals are compiled into a block-pattern fingerprint used to classify all future requests as guardrail block or LLM rejection via OR-combination of rules (timing is supplementary).

Key Contributions

  • First end-to-end black-box guardrail reconnaissance methodology requiring only 40 prompts and zero prior knowledge of the target.
  • Evaluation across 9 guardrails (ProtectAI, OpenAI, Meta, Microsoft, DuoGuard, LlamaFirewall, etc.), 6 block patterns, and 3 LLMs (GPT-4.1, Claude Sonnet 4.6, Gemini 2.5 Flash), yielding 162 unique target configurations.
  • 100% guardrail presence detection accuracy with statistically significant behavioral separation (\(q < 0.001\)).
  • Block-pattern fingerprint achieves 98% average F1 on 100 unseen prompts for distinguishing guardrail blocks from LLM rejections.
  • Correct attribution of content-category detection capability aligned with vendor-stated ground truth.

Results

  • Guardrail detection accuracy: 100% across all 162 configurations.
  • Behavioral separation: statistically significant at \(q < 0.001\) for HTTP and lexical signals.
  • Block vs. LLM rejection F1: 98% average on 100 unseen held-out prompts.
  • Detection capability attribution: correctly recovered the content categories each guardrail is designed to block (verified against vendor-stated capabilities in Table 3).
  • Note: The paper text is truncated before the per-guardrail breakdown tables; per-system numbers are not available in the provided excerpt.

Limitations

  • Threat model explicitly assumes no rate limiting or request throttling—a significant real-world constraint that would reduce the 40-prompt probe to a much slower operation or trigger detection.
  • Timing features are acknowledged as the "noisiest signals" due to network latency variance; they serve only a supplementary role in fingerprint classification.
  • Token count estimation uses a fixed heuristic (\(\approx 4\) chars/token) because the target tokenizer is unknown; this could degrade TPT reliability for non-English or code-heavy responses.
  • Malicious prompt-sets must avoid aggressively triggering LLM refusal (requirement 2 in Section 4.1), placing a design burden on prompt sourcing that may not generalize to all content categories.
  • Evaluation uses vendor-stated detection capabilities as ground truth, which may not reflect actual guardrail behavior at non-default thresholds.

Relevance to Agentic AI / LLM Agents

Guardrails are a primary safety layer in deployed agentic systems, and the inability to distinguish guardrail blocks from LLM refusals directly impairs realistic red-teaming and adversarial emulation of multi-step agents. This work enables security researchers to obtain precise feedback signals during agent penetration testing—feedback that is prerequisite for principled attack strategy selection (evasion perturbations vs. jailbreak prompts). For agentic pipeline defenders, the methodology also offers a tool to audit whether their own block patterns leak exploitable information. The strict black-box framing is directly aligned with production agentic deployment scenarios where system internals are opaque.