Skip to content

Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity

🕒 Published (v1): 2026-04-19 20:49 UTC · Source: Arxiv · link

Why this paper was selected

LLM agents fail to exploit unexpected discoveries; environmental curiosity gap exposed

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Current LLM-based agents systematically discover unexpected but relevant information in their environments yet fail to act on it—a gap the authors call environmental curiosity. The paper introduces solution injection as a diagnostic method and three new metrics (\(\text{discovery@k}\), \(\text{interaction@k}\), \(\text{pass@k}\)) to quantify this gap. Across three benchmarks and multiple models, even optimal inference-time configuration leaves agents ignoring injected solutions in the majority of runs.

Problem

Existing agentic benchmarks measure only task success, conflating agents that genuinely integrate observations with those executing fixed, training-conditioned patterns. There is no principled way to distinguish whether a failure is due to inability to find relevant information versus inability to act on it. Prior work assumes the agent loop faithfully processes unexpected observations, but this assumption is empirically untested.

Method

Solution Injection: The gold solution for each task is placed directly in the agent's environment—as an executable solution.sh in the working directory for Terminal-Bench and SWE-Bench, or as a documented CLI API endpoint (cli solution) for AppWorld. Solutions are labeled explicitly and guaranteed complete.

Three metrics are defined using the same unbiased \(\text{pass@k}\) estimator from Chen et al. (2021), with \(c_{\text{pass}}\), \(c_{\text{disc}}\), \(c_{\text{interact}}\) being per-task success counts over \(n\) attempts:

\[\text{pass@k} := \mathbb{E}\!\left[1 - \frac{\binom{n - c_{\text{pass}}}{k}}{\binom{n}{k}}\right]\]
\[\text{discovery@k} := \mathbb{E}\!\left[1 - \frac{\binom{n - c_{\text{disc}}}{k}}{\binom{n}{k}}\right], \quad \text{interaction@k} := \mathbb{E}\!\left[1 - \frac{\binom{n - c_{\text{interact}}}{k}}{\binom{n}{k}}\right]\]

Experiments use two scaffolds (Terminus, SWE-agent), two tool suites (bash-only vs. bash + str_replace_editor), three LLMs (gpt-oss-120b, GLM-4.7, command-a-reasoning fine-tunes), and \(n=10\) attempts per task. Fine-tuning experiments use rejection sampling on ~20k turns from three distributions: T-Bench-SFT, AppWorld-SFT, SWE-Bench-SFT.

Key Contributions

  • Environmental Curiosity formally defined and benchmarked: the capacity to recognize and investigate unexpected but relevant observations in response to environmental stimuli.
  • Solution Injection as a general benchmark augmentation applicable to any task with a gold solution.
  • \(\text{discovery@k}\) / \(\text{interaction@k}\) metrics that decouple finding from acting on information.
  • Systematic ablation of three test-time factors (tool availability, reasoning budget, prompting) showing all improve but none close the gap.
  • Evidence that narrow in-distribution SFT suppresses solution-space diversity, degrading \(\text{pass@k}\) scaling even on unmodified benchmarks.

Results

  • Terminal-Bench: \(\text{discovery@1}\) = 78.6–81.2%; \(\text{interaction@1}\) = 37.1–50.3%. Performance gain from injection: +11.4 pass@1 for gpt-oss-120b.
  • AppWorld: \(\text{discovery@1}\) > 90% for all models (gpt-oss: 97.54%); \(\text{interaction@1}\) ≤ 6.3% (gpt-oss: 0.53%). Performance gain: +2.6 pass@1.
  • SWE-Bench: \(\text{discovery@1}\) = 53.4–98.2%; \(\text{interaction@1}\) = 5.9–17.4%. Performance gain: +1.0 pass@1.
  • Tool availability: adding str_replace_editor improves pass@1 but reduces \(P(\text{interaction} | \text{discovery})\) in every configuration tested; bash-only roughly doubles interaction rates.
  • Reasoning budget: gpt-oss-120b Terminal-Bench \(\text{interaction@1}\) increases from 11% (low) → 37% (high); conditional \(P(\text{interact}|\text{discover})\) rises from 17.65% → 45.69%.
  • Prompting: explicit explore-first prompts improve original benchmarks by +2.57% on average; the best curiosity-inducing prompt is also the best-performing prompt on the unmodified benchmark.
  • Training distribution: AppWorld-SFT achieves higher \(\text{pass@1}\) on AppWorld (44.2 vs. 34.5) but lower \(\text{pass@10}\) (65.8 vs. 69.0) and lower \(\text{interaction@10}\) (26.9 vs. 41.5) compared to broader T-Bench-SFT.

Limitations

  • Even with all test-time factors jointly optimized (bash-only, high reasoning, explicit exploration prompts), agents still ignore injected solutions in the majority of trials.
  • Three SFT interventions aimed at inducing curiosity (rejection sampling for curious first turns, mid-trajectory file removal/re-addition, masked adversarial turns) all failed to improve interaction rates.
  • AppWorld interaction remains near zero regardless of reasoning level, suggesting task-type-specific resistance not addressed by compute scaling alone.
  • Environmental curiosity does not transfer across structurally distinct domains (Terminal-Bench vs. SWE-Bench), limiting the practical benefit of in-domain curiosity training.
  • Evaluated with only three LLMs; generalization to other model families is unconfirmed.
  • SWE-Bench results for cross-domain comparison use a single seed.

Relevance to Harnesses / Meta-Harnesses

The paper's three-layer agent decomposition—harness (Docker execution environment, solution evaluator), scaffold (ReACT loop, prompting, history management), tools—maps precisely onto the design space of meta-harnesses, and the paper shows that each layer independently controls a measurable behavioral property (environmental curiosity). The introduction of solution injection as a harness-level augmentation that is benchmark-agnostic demonstrates how harness design can create diagnostic probes without modifying the underlying task or model. The finding that tool availability (a scaffold/harness configuration choice) has the strongest and most consistent effect on interaction rates implies that meta-harness authors can directly modulate agent exploration behavior through infrastructure decisions alone—without any model changes. For harness builders, the practical upshot is that exposing richer tool surfaces (e.g., structured editors) may suppress exploratory observation-following even while improving average task performance, a tradeoff invisible to standard pass@k reporting.