Skip to content

STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

STRATUS is a multi-agent LLM system for autonomous Site Reliability Engineering (SRE) that covers the full failure-response pipeline—detection, diagnosis, and mitigation—on live cloud infrastructure. Its core contribution is Transactional Non-Regression (TNR), a formally specified safety property that enables safe undo-and-retry of mitigation actions. On two SRE benchmarks, STRATUS outperforms prior agentic SRE solutions by at least 1.5× in mitigation success rate.

Problem

Existing human-in-the-loop SRE cannot scale to the failure rates of modern clouds (hundreds of machine failures, thousands of disk failures per cluster). Prior AI/agentic tools assist human engineers on individual SRE subtasks (detection, RCA summarization) but do not autonomously mitigate failures. A key unsolved barrier is safety: an autonomous agent executing kubectl commands on a live system may worsen the already-degraded state, and no prior agentic SRE system provides formal guarantees against regression or recovery from failed mitigation attempts.

Method

STRATUS orchestrates four specialized LLM agents (Detection αD, Diagnosis αG, Mitigation αM, Undo αU) via a deterministic state machine using the CrewAI framework. Agents interact with the Kubernetes environment through a typed Action Space (Aread, Awrite, Aundo) with sandbox confinement enforcing read-only access for αD and αG.

The central mechanism is Transactional Non-Regression (TNR), formalized as an Alpern-Schneider safety property. TNR constructs transaction semantics around αM's mitigation plans: 1. Checkpoint: Record entry state spre before execution. 2. Execute: Run the action sequence (up to K=20 commands) under an Agent-Lock (readers-writer lock ensuring write exclusivity). 3. Commit/Abort: Commit if µ(spost) ≤ µ(spre) (severity does not increase); otherwise αU executes a stack-based undo to restore spre exactly.

The severity metric µ(s) = w1·|alerts| + w2·|SLA violations| + w3·|unhealthy nodes| provides a scalar health signal for commit/abort decisions. TNR guarantees that the externally visible severity sequence is monotonically non-increasing from the initial error baseline b, enabling safe iterative retrial of alternative mitigation plans. The undo mechanism leverages Kubernetes' state-reconciliation principle: resource state snapshots are pushed onto a stack and popped/reconciled in reverse order on abort.

Additional implementation details: NL2Kubectl translates natural-language commands to kubectl invocations; trace-based call-graph construction bootstraps αG's fault localization; multi-oracle termination combines alert clearance, request success rate, and component health.

Key Contributions

  • STRATUS system: End-to-end agentic SRE pipeline (detection → localization → RCA → mitigation) operating autonomously on live Kubernetes cloud systems.
  • Transactional Non-Regression (TNR): Formal safety specification with proof that externally visible severity never exceeds the initial error baseline; implemented via A-Lock, sandboxing, and stack-based undo.
  • Undo Agent (αU): A dedicated agent executing stack-based reconciliation-based rollback, enabling faithful state restoration after failed mitigation transactions.
  • Empirical evaluation on AIOpsLab (13 mitigation problems) and ITBench (18 mitigation problems) showing ≥1.5× improvement over state-of-the-art SRE agents across GPT-4o, GPT-4o-mini, and Llama 3.3.
  • Ablation establishing TNR necessity: retry without undo yields 23.1% success vs. 69.2% with full TNR on AIOpsLab.

Results

  • AIOpsLab mitigation (13 problems): STRATUS (GPT-4o) 69.2% vs. AOL-agent (GPT-4o) 46.2% (1.5× improvement); vs. ReAct 23.1%, Flash 38.5%.
  • ITBench mitigation (18 problems): STRATUS (GPT-4o) 50.0% vs. ITB-agent (GPT-4o) 9.2% (5.4× improvement).
  • Ablation (AIOpsLab, GPT-4o):
  • Full STRATUS: 69.2% success
  • No retry (single mitigation attempt): 15.4%
  • Naïve retry without undo: 23.1%
  • Detection (AIOpsLab, 32 problems): STRATUS (Llama) 93.8%, STRATUS (GPT-4o) 90.6%, outperforming all baselines.
  • Localization (28 problems): STRATUS (GPT-4o) 51.2%, highest among all agents.
  • RCA (26 problems): STRATUS (GPT-4o) 34.6%, second highest; low absolute numbers attributed to ambiguous benchmark label definitions.
  • STRATUS requires more time (811s vs. 223s for AOL-agent) and cost ($0.877 vs. $0.206) per problem due to multi-retry exploration; 80%+ of problems require at least one retry, 30%+ require ≥5 retries.

Limitations

  • Imperfect undo coverage: Application-specific states and external interactions (outside Kubernetes state-reconciliation) may not be fully recoverable; compensation logic for such cases is not automated.
  • Rule-based confinement may be incomplete: The sandboxing rules that prevent destructive actions may not capture all such actions in complex cloud environments.
  • No concurrent writer agents: A-Lock serializes all write agents; multi-tenant or high-throughput scenarios require the proposed but unimplemented Coordination Controller extension.
  • Pod-restart artifact in ITBench: 8 of 18 solved problems exploit a benchmark-specific behavior (injected faults disappearing on pod restart) that would not generalize to persistent real-world faults like misconfigurations or hardware failures.
  • Benchmark RCA metric ambiguity: AIOpsLab's label-matching RCA evaluation penalizes semantically correct but lexically different answers, making RCA success rates unreliable indicators.
  • High latency and cost: Multi-retry architecture is substantially slower and more expensive than single-pass agents, which may be prohibitive for latency-sensitive mitigation scenarios.

Relevance to Agentic AI / LLM Agents

STRATUS is a high-stakes deployment of agentic AI where the agent must act on live infrastructure with irreversible consequences, making it a critical case study for agentic safety in real-world tool-use settings. The TNR formalism addresses a fundamental open problem in agentic systems—how to grant write authority to an LLM-based agent without risking unrecoverable state—by borrowing transactional semantics from databases and adapting them to continuous-metric health monitoring. The undo-and-retry pattern directly enables exploration over a complex action space without catastrophic failure, a challenge shared by any agentic system operating in environments with side effects (code execution, browser automation, API calls). The paper's finding that naïve retry without undo yields only marginal gains (23.1% vs. 15.4%) while full TNR jumps to 69.2% quantifies the value of state-restoration for agentic robustness—a transferable insight for agent frameworks beyond SRE.