Skip to content

PRAXIS: Integrating Program Analysis with Observability for Root-Cause Analysis

🕒 Published (v1): 2025-12-26 18:56 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

PRAXIS is a 4-phase agentic orchestrator for cloud incident root-cause analysis (RCA) that combines LLM-driven graph traversal over a service dependency graph (SDG) and a hammock-block program dependence graph (PDG). By constraining LLM reasoning to incident-relevant dependency paths rather than monolithic code or unstructured prompts, it achieves 6.3× higher RCA accuracy and 5.3× lower token consumption vs. ReAct baselines. It is evaluated on a new 30-scenario benchmark (Code-Cloud-RCA, integrated into ITBench) running on live Kubernetes environments.

Problem

Cloud incidents caused by code or configuration faults (~24% of production outages not resolved by operational mitigation) require deep RCA beyond symptom-level observability signals. Existing ReAct-style SRE agents rely on unstructured, plain-text prompts over logs/traces and process code as monolithic text, leading to poor accuracy, high token consumption, and inability to trace fault propagation across service and code boundaries.

Method

PRAXIS operates in four phases orchestrated by a central controller:

  1. Data Gathering & Graph Construction: Collects alerts (Prometheus), distributed traces (Jaeger/OpenTelemetry), logs, metrics, and Kubernetes events. Constructs (a) a dynamic SDG from a real-time topology monitor and (b) a hammock-block PDG per microservice using Tree-sitter (AST parsing) and CLDK (data/call/control dependency analysis). Hammock blocks are SESE code regions that induce a module→class→function→statement hierarchy.

  2. Microservice Candidate Selection: LLM is prompted with the SDG and incident context (alerts + error traces) to select up to N=5 initial candidate microservices and initialize an investigation queue Q.

  3. RCA Decision-Making (iterative): For each entity dequeued from Q, PRAXIS:

  4. Selects an initial PDG node by matching observability context (stack traces, log strings) against PDG node attributes via ψmatch.
  5. Runs an agentic PDG traversal loop with 4 tools — Expand (go to parent hammock block), Relate (follow call/data/control edges), Complete (synthesize program context), Discard (prune irrelevant paths) — where the LLM sees only local block context at each step.
  6. Classifies the entity as PRIMARY FAILURE, SYMPTOM ONLY, or UNRELATED.
  7. Enqueues SDG dependees plus LLM-suggested entities (up to k=3) for subsequent investigation.

  8. Cross-SDG-PDG traversal handles multi-hop incidents where a code path in one microservice's PDG implicates a dependency in another microservice.

  9. Final RCA Summary: LLM synthesizes accumulated investigation history into a structured report with root-cause identification, fault propagation chain, and code-grounded evidence.

Key Contributions

  • PRAXIS orchestrator: structured LLM-driven dual-graph (SDG + PDG) agentic RCA workflow for cloud incidents
  • Application of hammock-block PDG for agentic RCA, enabling multi-granularity code traversal within a single uniform graph structure
  • Code-Cloud-RCA Benchmark: 30 Kubernetes-deployed scenarios covering software faults, configuration errors, deployment issues, and resource failures, integrated into ITBench
  • Cross-SDG-PDG traversal enabling multi-microservice, multi-hop RCA without independent agent combinations

Results

  • Root-cause reasoning accuracy: 61.5% (vs. ~9.8% for ReAct baseline) — 6.3× improvement
  • Root-cause identification accuracy: 73.9% (vs. ~21.8% for ReAct baseline) — 3.4× improvement
  • Token consumption per successful diagnosis: 166.5k tokens vs. 884.9k for baselines — 5.3× reduction
  • Evaluated across 30 real-world-derived cloud incident scenarios on a live Kubernetes cluster
  • Partial code availability still improves accuracy over observability-only mode; zero code availability degrades to baseline-level performance

Limitations

  • Requires microservice source code for PDG construction; efficacy degrades when source is unavailable
  • Static PDG (generated via Tree-sitter + CLDK) may not capture runtime-generated or dynamically loaded code paths
  • Benchmark scope is 30 scenarios on a single cloud application; broader generalizability is not demonstrated
  • Hammock-block PDG construction tool chain (Tree-sitter + CLDK) implies language support is bounded by those tools
  • Budget cap on PDG traversal steps and investigation queue size (N=5, k=3) may miss root causes in deeply nested or highly distributed incidents

Relevance to Harnesses / Meta-Harnesses

PRAXIS is a clear example of a structured agentic orchestrator / meta-harness: it coordinates multiple LLM invocations across a deterministic, graph-constrained workflow (candidate selection → iterative PDG traversal → role judgment → queue update → synthesis) rather than letting a single LLM agent free-range over unstructured context. Its design directly addresses the core meta-harness problem of context budget management — by exposing only local node neighborhoods to the LLM at each step, it prevents context overflow and reduces token waste by 5.3×. The cross-SDG-PDG traversal pattern, where the harness maintains a coherent global investigation narrative while dispatching focused sub-queries to the LLM, is a reusable architectural principle for any domain requiring hierarchical, multi-level dependency analysis. For harness researchers, the 4-tool traversal action space (Expand/Relate/Complete/Discard) and the queue-based orchestration loop are concrete, transferable primitives for constraining agentic exploration in structured knowledge graphs.