Skip to content

When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space

🕒 Published (v1): 2026-07-16 17:20 UTC · Source: Arxiv · link

Why this paper was selected

LLM-as-planner safety gap: linguistically safe → physically lethal; critical for embodied agent harnesses

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLMs used as embodied-agent planners face a safety problem distinct from text moderation: linguistically benign instructions (e.g., "microwave an egg") can be physically dangerous. This paper shows that content danger (CD) and physical danger (PD) form statistically separable directions in LLM hidden states, then exploits this to build PRISM—a single-layer linear probe that outperforms LLM judges and text guards on physical safety classification with far lower false-positive rates.

Problem

Existing text-safety guards (Llama Guard, LLM-as-judge) were designed to detect policy-violating wording, not physical-causality risk. Physically dangerous instructions often contain no harmful keywords, so content-oriented detectors either miss them entirely (Llama Guard 3-1B: 0% recall on PSB-1K) or recover recall only by massively over-blocking safe tasks (Qwen2.5-32B judge: 37.0% FPR). Hidden-state prototype methods (LPM) also fail to transfer from CD to PD, detecting only 9.6–19.3% of physical-danger OOD samples. No prior work established whether CD and PD are even representationally distinct problems.

Method

Hidden-state direction analysis. For each of three classes (safe \(S\), content-danger \(E\), physical-danger \(I\)), class-mean hidden states are computed at layer \(L\): $\(\mu_\text{CD} = \frac{1}{|E|}\sum_{i\in E} h_L(x_i), \quad \mu_\text{PD} = \frac{1}{|I|}\sum_{i\in I} h_L(x_i)\)$ The Content Danger Direction \(\text{CDD} = \mu_\text{CD} - \mu_\text{safe}\) and Physical Danger Direction \(\text{PDD} = \mu_\text{PD} - \mu_\text{safe}\) are estimated. Two angular metrics quantify separation: the mean-difference angle \(\theta_\text{md}\) (centroid geometry) and the probe-weight angle \(\theta_\text{pw}\) (discriminative geometry from two separate L2-regularized logistic probes). A random-split null is constructed by randomly partitioning the 300 unsafe examples into same-sized subsets 200 times and fitting the identical pipeline, yielding a null distribution for \(\theta_\text{pw}\).

PRISM. A single-layer L2-regularized logistic probe trained on the full hidden state \(h_L(x) \in \mathbb{R}^d\) with all three classes collapsed to a binary safe/unsafe label: $\(p(y=1|x) = \sigma(w^\top h_L(x) + b), \quad \hat{y} = \mathbf{1}[p \geq \tau]\)$ Layer \(\ell^*\) is selected by per-layer 5-fold CV on accuracy. Features are standardized fold-internally. Training jointly covers CD and PD examples, so the single boundary must separate both danger types from safe tasks.

PSB-1K benchmark. 1,000 minimally contrasted safe/physical-risk pairs across ten household risk families (microwave metal contact, liquid contact with powered devices, open-flame proximity, etc.), with unsafe instructions deliberately avoiding harm keywords (e.g., "danger," "burn," "shock"). Evaluated with pair-wise GroupKFold to prevent leakage.

Key Contributions

  • Demonstration via random-split null tests that CD and PD form statistically separable hidden-state directions across Qwen2.5-3B/7B/14B/32B, Phi-3.5-mini, and SmolLM2-1.7B (\(\theta_\text{pw} = 75.9^\circ\)–\(71.9^\circ\) vs. null mean \(60.35^\circ \pm 1.58^\circ\), \(z=10.09\), \(p<0.0001\) at 3B).
  • PRISM: a lightweight, model-agnostic linear probe over frozen LLM middle-to-late hidden states for joint CD+PD safety classification.
  • PhysicalSafetyBench-1K (PSB-1K): a 2,000-example contrastive benchmark targeting physically grounded danger without explicit harm wording, covering ten household risk families.
  • Evaluation protocol spanning SafeAgentBench, adapted SafeText, EARBench, and PSB-1K with explicit CD/PD subtype reporting and latency profiling.

Results

SafeAgentBench (600 tasks, 5-fold CV): - PRISM accuracy: 86.2–87.7%; F1: 0.861–0.876; AUC: 0.926–0.950; FPR: 11.7–13.7%; PD recall: 79.1–82.4%; CD recall: ≥93.8% (Qwen2.5-3B–32B). - LLM judge (same-scale Qwen2.5-Instruct) FPR: 24.7–39.0% — 2–3× higher than PRISM. - CDD-only probe: PD OOD detection collapses to 19.8–32.6% across scales. - LPM: PD OOD detection only 9.6–19.3% across scales. - PRISM latency speedup over judge: 1.86–2.12× at 7B/14B/32B (56.8/146.1/103.4 ms vs. 116.8/309.1/191.9 ms per sample).

PSB-1K (2,000 examples; no harm keywords): - PRISM (Qwen2.5-3B): 99.6% accuracy, 99.9% PD recall, 0.7% FPR. - Llama Guard 3-1B: 0/1,000 true positives (0.0% PD recall, 0.0% FPR — completely blind to physical danger). - Qwen2.5-3B judge: high unsafe recall but 67.8% FPR on safe tasks.

Limitations

  • Random-split null is formally validated only for Qwen2.5-3B; the 7B/14B/32B cross-model pattern is reported as observed angles without completed null tests.
  • SafeAgentBench CD/PD split uses a 25-keyword deterministic rule with known leakage (e.g., "drop" appears in 12 safe and 29 CD cases), retained for comparability but not leakage-free.
  • PRISM's selected-layer protocol uses per-layer 5-fold CV on the training set without a fully nested layer-selection protocol, which may inflate layer-selection optimism.
  • PSB-1K is self-constructed and currently covers ten household risk families; generalization to non-household or industrial physical domains is untested.
  • PRISM is diagnostic (pre-execution classification), not a grounding-aware monitor; it relies on the planning LLM's hidden states and does not perceive the actual physical environment.
  • Cross-architecture replication on Phi-3.5-mini and SmolLM2-1.7B is presented as supporting evidence, not a full experimental study with the same depth as Qwen2.5.

Relevance to Agentic AI / LLM Agents

This paper directly addresses a core safety gap in LLM-based embodied agent pipelines: the planner's text-level safety guardrails cannot catch physical-causality risks, creating a silent failure mode where safe-looking instructions trigger dangerous robot actions. The PRISM probe is immediately deployable as a pre-execution filter in modular LLM+VLM or VLA architectures without fine-tuning or additional LLM inference, making it practically relevant for on-device embodied agents. The CD/PD separability finding also has theoretical implications for the representation engineering literature — it shows that "unsafe" is not a single hidden-state direction but at least a two-signal space in agentic contexts, which matters for alignment and refusal steering work. PSB-1K fills a concrete benchmark gap for the community evaluating household robot safety without keyword-based shortcuts.