Near-Miss: Latent Policy Failure Detection in Agentic Workflows¶

🕒 Published (v1): 2026-03-31 12:26 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Current agentic workflow evaluations compare only final system state against ground truth, missing a subtle failure class where agents skip required policy checks yet reach correct outcomes by chance. This paper defines and operationalizes near-misses (latent failures): trajectories where the gold state is matched but a required information-gathering tool call was never made. Latent failures occur in 8–17% of successfully completed trajectories across six contemporary LLMs on the τ²-verified Airlines benchmark.

Problem¶

Reference-based evaluation of LLM agents in business process automation only checks whether the final environment state matches a predefined gold state. This misses cases where an agent bypasses mandatory policy verification steps (e.g., checking reservation timestamps, flight availability, or user membership) but still produces the correct outcome due to benign user input or favorable data. These near-misses are invisible to standard benchmarks yet represent real security and compliance risks under adversarial conditions.

Method¶

The approach builds on the ToolGuard framework, which compiles natural-language policy documents offline into deterministic Python "guard" functions (\(\text{GF}\)) keyed to each mutating tool call (MTC). Each guard internally calls read-only data-access tools (\(\text{RO} = \{ro_1, \ldots, ro_k\}\)) to verify policy conditions.

Near-miss definition: A mutating tool call \(\text{MTC}(\text{args})\) is a near-miss (\(\text{NM} = \text{TRUE}\)) if no equivalent read-only tool call \(ro_i \in \text{RO}\) appears in the agent's trajectory prior to MTC — i.e., the required information was never actually retrieved.

Detection pipeline (offline + runtime): 1. For each MTC in a completed trajectory, load and execute the corresponding guard code. 2. When the guard code invokes a read-only tool \(ro\), search the trajectory history for \(ro\) or any functionally equivalent alternative. 3. Two history search strategies are compared: - (2.1) LLM-based search: prompt an LLM to identify whether the information need is satisfied by any prior call. - (2.2) Code-generation-based search: generate offline Python code that deterministically checks for the presence of \(ro\) or equivalents in the trajectory; no LLM at runtime.

Strategy (2.2) with Claude-Sonnet4 as code generator achieves perfect precision/recall and is adopted for main experiments. Runtime near-miss detection requires only code execution — zero LLM overhead after the offline build phase.

Key Contributions¶

Formal definition of near-miss / latent failure as a property of mutating tool calls in agentic trajectories, independent of final outcome.
A novel near-miss rate (NMR) metric that quantifies policy-check bypassing even in ostensibly successful runs.
Integration with ToolGuard's guard-code infrastructure to enable deterministic, low-overhead latent failure detection.
Empirical NMR measurements across six LLMs (three proprietary, three open) on τ²-verified Airlines, exposing a systematic blind spot in existing evaluation methodology.
Analysis identifying which mutating tools (update_reservation_flights) and which omitted read-only tools (get_flight_status, get_user_details) are most implicated.

Results¶

Near-miss rates (computed over trajectories containing at least one MTC):
GPT5-chat: 17.3%
Claude-Sonnet4: 12.1%
Gemini-3-pro: 14.0%
GPT-oss-120b: 8.6% (best)
Qwen2.5-72b: 8.4% (caveat: 144/200 trajectories invoke MTCs due to over-eagerness)
Kimi-K2.5: 14.4%
Policy violations account for ~25% of all simulation failures in τ²-verified Airlines.
Code-generation-based search (Claude-Sonnet4 as generator) achieves P=1.00, R=1.00 on the 400-trajectory annotated subset; GPT5.1-Codex as generator shows many false positives (P=0.17–0.20).
LLM-based search achieves reasonable but imperfect precision/recall (P=0.71–1.00, R=0.74–1.00).
Inter-annotator agreement on the 50-trajectory subset was "almost perfect."
No clear correlation observed between explicit policy violation ratio and near-miss ratio across models.

Limitations¶

Evaluated on a single benchmark domain (τ²-verified Airlines); generalizability to Retail, Telecom, or other domains is asserted but not demonstrated.
Requires pre-existing ToolGuard guard code for each tool; domains without compiled guards cannot be analyzed without the offline build phase.
Assumes all necessary data-access tools exist and are available to the agent; the authors had to add two missing tools (get_flight_status, get_flight_instance) to the benchmark.
The NMR metric cannot detect latent failures for policy items that do not require any read-only tool invocations.
Human annotation of near-miss ground truth was performed by paper authors, introducing potential bias.
The approach only covers policies governing mutating tool calls; policies about information disclosure, customer confirmation, etc. are explicitly out of scope.

Relevance to Harnesses / Meta-Harnesses¶

This paper demonstrates a two-phase harness pattern — offline policy compilation (ToolGuard guard-code generation) paired with a runtime trajectory-analysis harness — that extends standard outcome-based evaluation into process-level compliance checking. For researchers building meta-harnesses that orchestrate and evaluate agentic pipelines, the near-miss metric is a concrete example of a post-hoc trajectory analysis layer that can be composed on top of any existing benchmark runner without modifying the agent under test. The generated-code-based history search achieves zero runtime LLM cost, making it directly embeddable as a lightweight evaluation module in harness infrastructure. The work also highlights that harnesses relying solely on final-state gold-truth comparison may silently pass flawed agent behaviors, motivating richer in-trajectory instrumentation as a first-class harness capability.