Skip to content

EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents

🕒 Published (v1): 2026-05-29 07:13 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML; hazard-aware guardrails for safe action planning in embodied agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EMBGuard is the first MLLM-based safety guardrail specifically designed for embodied agents, decoupling physical risk reasoning from agent policy by evaluating (visual observation, action) pairs. Accompanying it are EMBHazard (15.1K training pairs) and EMBGuardTest (329 manually curated real-world samples) spanning seven physical risk categories. Despite being only 2B–4B parameters, EMBGuard matches proprietary models like GPT-5.1 and Gemini-2.5-Pro at 0.5–0.7s per sample.

Problem

MLLM-powered embodied agents executing long-horizon tasks in physical environments lack explicit mechanisms for action-conditioned hazard detection. Delegating safety reasoning entirely to the planning agent causes two failure modes: missing risky interactions when the agent is focused on task completion, or over-identifying risks when safety is prioritized. No prior work had built a concrete guardrail implementation for embodied (physical) agents, as opposed to digital/LLM guardrails for toxic text.

Method

EMBGuard is formalized as a function \(R : (I, a) \rightarrow (r_{\text{bin}}, r_{\text{type}}, h)\), mapping a scene image \(I\) and candidate action \(a\) to a binary risk label, a risk category from a seven-class taxonomy, and a natural-language hazard description \(h\).

Dataset construction (EMBHazard): A three-stage pipeline generates training data: 1. Risk-driven scenario generation: 24 risk-inducing patterns are defined from WHO ICD-11 and CPSC NEISS; experts seed 2.4K text scenarios (risk type, hazard description, action triplets). 2. Compositional scenario diversification: Hazards are represented as scene graphs; four graph-level/action-level transformations — scene augmentation \(f_{\text{scene}}\), hazard addition \(f_{\text{hazard}}^+\), action modification \(f_{\text{action}}\), and hazard removal \(f_{\text{hazard}}^-\) — generate four variant types (Causal Risky, Selective Risky, Decoupled Benign, Absent Benign), yielding 17K augmented scenarios. 3. Image generation and VQA filtering: Gemini-3-pro-image generates photorealistic images from scene graph descriptions; GPT-5.1 runs VQA against hazard subgraph edges to filter images that fail to preserve critical spatial relationships.

EMBGuardTest: 329 real-world images independently annotated by five authors across all risk/scenario types, cross-validated, with no overlap with training data.

Model training: Supervised fine-tuning of Qwen-3-VL (2B and 4B) on EMBHazard via LLaMAFactory (lr=1e-5, 4 epochs, 8× A6000 GPUs), with the vision encoder frozen to preserve hazard-explanation capability.

Key Contributions

  • EMBHazard: 15.1K (image, action) pairs with 8.7K photorealistic images (7.8K risky, 7.3K benign) covering 7 physical risk categories across 4 compositional scenario types.
  • EMBGuardTest: 329 manually curated real-world benchmark samples with cross-validated annotations and no synthetic/training data overlap.
  • EMBGuard model: First purpose-built safety guardrail for embodied agents, operating on (observation, action) pairs to assess risk, classify risk type, and explain the hazardous configuration.
  • Seven-category physical risk taxonomy grounded in WHO ICD-11 and CPSC NEISS incident data.
  • Scene-graph-based compositional augmentation pipeline enabling controllable hazard/action manipulation.

Results

  • Potential Risk Accuracy on EMBGuardTest: EMBGuard-4B 54.3%, EMBGuard-2B 51.6%, vs. Gemini-2.5-Pro 58.4%, GPT-5.1 55.8%, Qwen-3-VL-32B 49.7%.
  • Potential Risk Accuracy on Held-out Set: EMBGuard-4B 71.2%, EMBGuard-2B 68.3%, vs. Gemini-2.5-Pro 61.4%, GPT-5.1 69.1%, Qwen-3-VL-32B 71.8%.
  • Risk Type Accuracy (conditional on correct binary prediction) on EMBGuardTest: EMBGuard-4B 50.3% vs. GPT-5.1 58.1%, Gemini-2.5-Pro 56.8%.
  • Hazard Accuracy on Held-out Set: EMBGuard-4B 50.1%, matching or exceeding GPT-5.1 (57.0%) and Gemini-2.5-Pro (63.8%) only on some splits; full parity not achieved on hazard explanation.
  • False positives: Gemini-2.5-Pro misclassifies 83.3% of benign scenarios as risky; EMBGuard significantly reduces this over-conservative bias.
  • Inference speed: 0.535s/sample (2B) and 0.719s/sample (4B) on a single RTX 6000 Ada GPU.
  • Synthetic-to-real correlation: Strong Pearson correlation (\(p < 0.05\)) between Held-out Set and EMBGuardTest performance, validating the synthetic pipeline.
  • All models show systematic under-detection of crush, contamination, and chemical risks (requiring causal/temporal reasoning) compared to perceptually salient hazards (fire, electrical, slip, cut).

Limitations

  • Hazard Accuracy (free-form explanation quality) remains substantially lower than binary or category-level metrics across all models, including EMBGuard.
  • Overall Potential Risk Accuracy peaks at ~71% even on the held-out synthetic set, indicating the task is far from solved.
  • Freezing the vision encoder (necessary to preserve explanation quality in small models) may cap visual understanding ceiling.
  • Dataset is currently limited to 329 real-world evaluation samples; broader diversity of real environments is not yet covered.
  • The paper's safe-planning integration experiments are referenced but results are truncated in the provided text.
  • Models still show systematic bias toward perceptually salient risk types; causal/temporal hazards (contamination, chemical, crush) remain underdetected.

Relevance to Agentic AI / LLM Agents

EMBGuard directly addresses a critical open problem in agentic AI: how to enforce safety constraints on an embodied agent's action stream without burdening the planning agent itself. The guardrail-as-separate-module design mirrors the growing architectural pattern of decoupled policy and critic modules in LLM agent systems, and the (observation, action) input interface makes it plug-compatible with any MLLM-based planner. The action-conditioned risk framing — where hazard + action together determine danger, not environment state alone — is a conceptual advance over static scene-safety classifiers, and the compositional dataset methodology (scene graphs, controlled augmentation) offers a replicable template for generating safety training data in other agentic domains. The demonstrated false-positive reduction is especially important for deployable agents, where over-refusal is as much a failure mode as under-refusal.