Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security¶
🕒 Published (v1): 2026-05-17 10:26 UTC · Source: Arxiv · link
Why this paper was selected
Comprehensive survey of safety, robustness, privacy, security in agentic AI systems
Ask a follow-up
Open an assistant pre-loaded with this paper's context.
💬 Ask ChatGPT✦ Ask Claude
TL;DR¶
A structured survey on trustworthy agentic AI that organizes risks and mitigations around a five-stage workflow lens (Perceive→Plan→Act→Reflect→Learn), focusing on two trust dimensions—Safety & Robustness and Privacy & System Security. It consolidates fragmented evaluation literature into a unified metrics-and-benchmarks hub with scenario-to-metric release gating guidance. A case study on real-world security failures in open-source agentic systems (OpenClaw/Moltbook) grounds abstract taxonomy in concrete incidents.
Problem¶
Prior surveys on trustworthy AI address either general LLM safety (without workflow specificity) or agent architectures (without a multi-dimensional trust taxonomy and practical release-gating). Practitioners lack a consistent framework for mapping threats to mitigations at each agent lifecycle stage, and evaluation signals (outcome vs. process metrics) are fragmented across sub-fields, making cross-system comparison and deployment gating difficult.
Method¶
The survey adopts an agent workflow lens—Perceive → Plan → Act → Reflect → Learn—as a consistent interface for mapping threats and defenses across architectures. Each of the two core trust dimensions (Safety & Robustness; Privacy & System Security) is analyzed using a uniform Definition → Risks → Methods structure keyed to workflow stages. Five assurance hooks are defined at stage boundaries:
| Hook | Stage | Mechanism |
|---|---|---|
| Validation | Perceive | Input filters, jailbreak detection, RAG citation checks |
| Verification | Plan | Formal/logic verification of plans against constraints \(K\) |
| Safeguards | Act | Runtime shields, permission gates, tool sandboxing |
| Diagnostics | Reflect | Anomaly detection (OOD/drift), value-alignment checks |
| Logging | All | Tamper-evident traces for audit and accountability |
An assurance stack spanning four tiers—pre-deployment (red teaming, hazard analysis), training-time (constrained/safe RL, preference optimization), runtime (shielding, rollback, DLP hooks), and post-deployment (audit)—provides defense-in-depth. The agent is formalized as an MDP \(M = (S, A, P, r, \gamma)\), extended to POMDP under partial observability; safe RL is framed as a constrained MDP (CMDP): \(\max_\pi J(\pi) \text{ s.t. } J_{c_i}(\pi) \leq d_i\).
Key Contributions¶
- Two-dimension trust taxonomy (Safety+Robustness; Privacy+System Security) with the remaining trust axes (alignment, transparency, fairness, accountability) treated as supporting context rather than separate chapters.
- Risk-to-method mapping along the five-stage workflow, with five named assurance hooks as formal control points.
- Consolidated evaluation hub: metric dictionary covering both outcome metrics (task success rate, catastrophic-event rate) and process metrics (constraint violation rate, trace completeness, adversarial success rate), plus scenario-to-metric release gating guidance.
- Classification of RL/preference-optimization training families (PPO, offline RL, HRL, MBRL, RLHF/DPO) by their specific trust implications.
- Case study on real-world security failures in OpenClaw/Moltbook open-source agentic systems.
- Enumeration of open challenges: self-evolving agents under concept drift, runtime monitoring for rare events, privacy-preserving personalization, and the trust–utility trade-off.
Results¶
This is a survey paper; no original empirical benchmark numbers appear in the provided text. The comparative survey taxonomy (Table 1) shows that this work is the only surveyed paper achieving all four properties: Multi-Dimensional Trust taxonomy (MDT), Workflow Lens (WL), Evaluation Hub (EH), and Release Gating (RG)—where all prior surveys miss at least one.
Limitations¶
- Authors explicitly disclaim exhaustiveness: risks and mitigations are "illustrative examples" of current knowledge, not a provably complete enumeration.
- Formal multi-agent RL treatment is explicitly out of scope; multi-agent trustworthiness risks are discussed only where directly relevant.
- Two trustworthiness dimensions are treated as full chapters; value alignment, transparency, fairness, and accountability receive only contextual discussion.
- The provided text is truncated before the evaluation hub details and the OpenClaw/Moltbook case study, so quantitative claims from those sections cannot be verified from this excerpt.
Relevance to Harnesses / Meta-Harnesses¶
The five assurance hooks (Validation, Verification, Safeguards, Diagnostics, Logging) map directly onto what a meta-harness implements as lifecycle wrappers around agent execution—pre-call input sanitization, post-plan constraint checking, sandboxed tool dispatch, outcome-level anomaly detection, and append-only audit logging. The layered assurance stack (pre-deployment → training-time → runtime → post-deployment) is structurally equivalent to a meta-harness design pattern, where each tier compensates for gaps left by the one above it. The scenario-to-metric release gating guidance in the evaluation hub directly codifies what a quality gate in a production harness should check before allowing agent output to propagate downstream. Practitioners building trustworthy harnesses can use this survey's hook taxonomy and metric dictionary as a specification checklist for which control points to instrument.