AgentAbstain: Do LLM Agents Know When Not to Act?

🕒 Published (v1): 2026-07-11 00:57 UTC · Source: Arxiv · link

Why this paper was selected

Abstention/self-awareness in agents; critical eval dimension missing from most harness benchmarks

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgentAbstain introduces the first systematic evaluation framework for agentic abstention — the calibrated ability of tool-using LLM agents to recognize when not to act. It pairs a 263-task benchmark (AGENTABSTAIN) spanning 8 abstention scenarios in 42 sandboxed MCP environments with ABSTAINGEN, a fully automated pipeline for synthesizing those tasks end-to-end. Across 17 frontier LLMs run through 4 agent harnesses, the best model achieves only 59.5% Paired Accuracy, and abstention skill is largely orthogonal to general task-solving skill.

Problem

Existing agent benchmarks measure task success but not abstention: whether agents correctly withhold action under ambiguity, conflicting constraints, tool failures, or emergent risks. This creates deployment risk because agents may commit irreversible actions (e.g., canceling a flight reservation) before or instead of recognizing that execution is inappropriate. Prior abstention work targets single-turn QA with no environment side effects; safety benchmarks address explicitly malicious goals rather than legitimately-intended-but-incorrectly-executable tasks; tool-reliability benchmarks test correct invocation, not appropriate non-invocation; and clarification-seeking work covers only the pre-execution decision.

Method

The framework has four components:

Taxonomy: 8 abstention scenarios (S1–S8) partitioned into pre-execution (trigger detectable from instruction \(q\) or tool inventory \(\tau\) alone) and runtime (trigger emerges only through tool interaction).

Paired task model: Every benchmark instance is a pair \((T^+, T^-)\) sharing the same sandbox environment but differing in exactly one controlled perturbation \(\delta\) applied to \(q\), environment state \(e\), or \(\tau\). \(T^+\) is a fully specified should-act task; \(T^-\) introduces a single trigger making execution incorrect, harmful, or epistemically unjustified. By construction, constant policies (always-act or always-refuse) cannot exceed 50% Paired Accuracy, so discriminating the pair is required.

Composite evaluation: Each task is scored via (a) a deterministic commit check on the tool-call trace (did a state-mutating commit-class tool fire?) and (b) an LLM judge (GPT-5.4) on the terminal response. For operational tasks (\(\alpha = \text{OP}\)) both signals must agree; for informational tasks (\(\alpha = \text{INFO}\), deliverable is the response) the judge verdict alone determines the pass.

ABSTAINGEN construction pipeline: Two-layer automated system. Layer 1 takes raw seed instructions from 8 upstream benchmarks, clusters tool capabilities via greedy set-cover, synthesizes MCP environment schemas, and generates environment code via per-environment coding agents (validated by automated import, callability, and round-trip checks). Layer 2 generates the act task (instruction, system prompt, execution DAG \(G^+\)) then derives the abstain variant by applying minimal perturbation \(\delta\). Three orthogonal verification mechanisms gate every stage: programmatic validators on DAG topology and perturbation locus, a cross-family LLM rubric judge, and a cross-family falsification critic that actively searches for counterexamples to the abstain claim. A construction-time DAG replay executes \(G^+\) against the live environment and rejects failing pairs.

Experiments run 17 frontier LLMs across 4 harnesses — OpenAI Agents SDK, Anthropic Claude SDK, Google ADK, and OpenClaw — each given the same MCP tool exposure.

Key Contributions

• First agent-native abstention taxonomy: 8 scenarios organized by when (pre-execution vs. runtime) and where (query / environment state / tools) the trigger becomes observable
• AGENTABSTAIN benchmark: 263 paired tasks (526 total) in 42 MCP sandbox environments with composite commit-check + LLM-judge evaluation
• ABSTAINGEN: fully automated, scalable construction pipeline supporting on-demand task regeneration to mitigate data contamination as models evolve; 94–98% human-rated validity on a 100-pair stratified sample
• Empirical finding that best frontier model (Gemini 3.1 Pro) reaches only 59.5% Paired Accuracy and that abstention capability scales independently of task-solving capability
• Identification of post-hoc abstention as a failure mode unique to agentic settings: agent commits an irreversible action and only then verbally acknowledges the problem

Results

• Best overall: Gemini 3.1 Pro, 59.5% Paired Accuracy; second: Claude Opus 4.7, 59.4% (gap < 0.2 pp)
• Mean Paired Accuracy across 17 models: 45.7%; 13 of 17 models fall below 50%
• Lowest performers: GPT-4o (33.0%), Kimi K2.5 (33.4%), DeepSeek V4 Pro (36.9%)
• Mean act accuracy 80.6% vs. mean abstain accuracy 59.1% — a 21-point gap persistent across all models
• Claude Opus 4.7 is the only model whose abstain accuracy (79.0%) exceeds its act accuracy; it achieves top-2 Paired Accuracy through an abstain-dominant profile, while Gemini 3.1 Pro achieves top-1 through the highest act accuracy in the top-5 (90.5%) paired with moderate abstain accuracy (65.4%)
• Human annotators rated 96%, 94%, and 98% of sampled task pairs as well-designed with high inter-annotator agreement

Limitations

• Only 30–39 pairs per scenario; limited statistical power for per-category conclusions
• Each task executed once per model; stochastic variance across runs not quantified
• LLM judge (GPT-5.4) introduces potential evaluation bias; judgment quality is not ablated against additional human labels
• All 42 environments are synthetically generated; real-world tool diversity and state complexity may not be fully captured
• Abstention is operationally defined as explicit refusal or clarification-request at termination; implicit abstention (halting without signaling) scores as failure, which may undercount cautious behavior in some deployment settings
• Concurrent work [34] on the same topic is compared only briefly

Relevance to Harnesses / Meta-Harnesses

The paper directly treats 4 agent harnesses (OpenAI Agents SDK, Anthropic Claude SDK, Google ADK, OpenClaw) as first-class experimental variables under identical MCP tool exposure, making it a cross-harness comparative study of a capability dimension — abstention — that harness-level scaffolding could plausibly influence. ABSTAINGEN is itself a meta-harness: a multi-stage automated pipeline that orchestrates coding agents, LLM rubric judges, and falsification critics to synthesize MCP environments and generate validated benchmark tasks on demand, with on-demand regeneration as an explicit contamination-mitigation mechanism. The composite evaluation architecture — harness-level tool-call trace capture (commit check) composed with model-level judgment (LLM judge) — demonstrates how harness observability can be layered with external evaluation to catch failure modes invisible to either signal alone. For researchers tracking harnesses/meta-harnesses, the benchmark establishes that harness choice is a distinct experimental variable warranting dedicated abstention-aware evaluation, not subsumed by task-success metrics.