Skip to content

Agentra: A Supervisable Multi-Agent Framework for Enterprise Intrusion Response

🕒 Published (v1): 2026-06-16 17:17 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Agentra is a supervisable multi-agent intrusion response framework that converts IDS/EDR/XDR alerts into OASIS CACAO v2.0 playbooks grounded in MITRE ATT&CK, D3FEND, and NIST CSF 2.0. It interposes a deterministic Orchestrator over a bounded Planner–Validator review loop, a Moderator security gateway for retrieval sanitization, and a gated Executor with append-only audit logging. On a 120-event corpus, the full stack raises FP-aware IRS F1 from 0.61 (static CACAO baseline) to 0.84 while restoring the harmful-action rate to 0.0%.

Problem

Enterprise security response still depends on static, analyst-maintained SOAR playbooks that cannot adapt when adversaries vary tactics or chain actions across surfaces. LLM-assisted response systems (e.g., IRCopilot) lack the layered supervision—independent plan review, mediated retrieval, bounded action scope, and integrity-protected audit—required for enterprise deployment. Indirect prompt injection through retrieved threat intelligence (demonstrated by Greshake et al.) and tool-using agent vulnerabilities (AgentDojo) mean retrieved content and model outputs must be treated as untrusted inputs.

Method

Agentra formalizes response as Algorithm 1: alert evidence \(E\) is normalized into an enriched Incident Context \(I\); Graph-RAG internal retrieval (\(X_\text{int}\)) and web/OSINT external retrieval (\(X_\text{ext}\)) are screened by a Moderator agent that strips indirect injection, allow-list violations, and hallucinated D3FEND identifiers before exposing sanitized context \(X\) to the Planner.

The Planner drafts a candidate IRP \(P\) composed of discrete actions \(a \in P\) but has no execution access. A Validator pool \(V = \{v_1, \ldots, v_n\}\) reviews \(P\) independently; each \(v_i\) returns a binary vote \(b_i \in \{0,1\}\) and rationale \(r_i\). The plan advances only if \(\sum b_i \geq k\); rejected plans return to the Planner for bounded revision (cap \(R\)), otherwise the case escalates. Post-analyst-approval, each action is gated by (1) membership in a bounded Action Catalog \(C\), (2) a Sanity Checker for dependency consistency, and (3) a risk score \(\text{Risk}(a) = f(\kappa(a), \beta(a), \rho(a))\) mapping to authorization tiers \(\tau \in \{\text{advise}, \text{approve}, \text{senior-approve}\}\). A StateChanged guard invalidates the plan if the asset state drifts between planning and execution. All decisions are recorded in a Merkle-tree-chained append-only audit ledger.

The ablation (A0–A4) adds components incrementally over the static CACAO baseline; five Planner/Validator combinations (M1–M5) span local models (Qwen3-32B, Mistral-Small-3.2-24B, Foundation-Sec-8B) and API models (Claude Sonnet 4.6, GPT-5) in replace-CACAO or enhance-CACAO modes, served via vLLM FP8 on a single H200 NVL.

Key Contributions

  • Agentra framework: alert → ontology-grounded CACAO v2.0 IRP pipeline with traceable D3FEND mitigation tags.
  • Bounded Planner–Validator review loop formalizing \(k\)-of-\(n\) approval with structured feedback and escalation.
  • Moderator-based Security Gateway screening retrievals for injection, allow-list violations, hallucinated ontology IDs, and data/tool poisoning.
  • Merkle-tree append-only audit substrate making deletion or retroactive modification detectable.
  • Five-step additive ablation on a 120-event corpus (ThreatHunter-Playbook + Splunk BOTSv3 + DARPA OpTC) isolating the marginal contribution of each architectural gate.

Results

  • Static CACAO baseline (A0): recall 0.55, F1 = 0.61, FP-rate = 0.50, harmful-action rate = 0.0%.
  • A1 (Planner only): recall rises sharply (M4/Claude Sonnet: 0.55→0.90; M5/GPT-5: 0.91), but D3FEND precision collapses (\(\text{PrecD3} \leq 0.30\) for all combinations) and harmful-action rate reaches 14–16% for API planners; M4 FP-rate = 0.97.
  • A2 (Validator added): combination-dependent—reduces FP-rate for M1 to 0.20 but collapses M5 recall to 0.14 (strict Sonnet Validator rejects most GPT-5 proposals within the 600 s debate budget).
  • A4 (full stack, best combination): FP-aware IRS F1 = 0.84; harmful-action rate restored to 0.0%; A3/A4 add controlled CTI and Graph-RAG grounding that lifts D3FEND precision and coverage while re-stabilizing safety.
  • Corpus: 58 malicious events (43 ATT&CK techniques/sub-techniques) + 62 benign/FP stress cases; single seed (seed 42); multi-seed significance testing deferred.

Limitations

  • Single-seed evaluation (seed 42); no McNemar significance testing; point estimates only.
  • Validator pool evaluated as a single Planner–Validator pair, not a true \(k\)-of-\(n\) heterogeneous quorum; larger pools deferred to future work.
  • Moderator prompt-injection robustness is a designed control, not validated against an adaptive adversarial retrieval suite.
  • Harmful-action rate is a conservative analytical overlay, not a live execution measurement; no live enterprise deployment.
  • 120-event corpus; dataset-recognition risk not fully eliminated despite anonymization.
  • Byzantine independence assumption is an analogy only—shared base models do not satisfy classical independent-failure properties.
  • Risk scoring uses discretized static service-dependency overlay with fixed thresholds, not dynamic asset graphs.

Relevance to Harnesses / Meta-Harnesses

Agentra is a production-oriented multi-agent orchestration harness in which a deterministic Orchestrator drives a bounded Planner–Validator review loop—a pattern directly analogous to meta-harness designs that wrap generative agents with structured verification, gate-keeping, and audit layers. The additive ablation (A0→A4) is itself a harness evaluation methodology: it isolates the marginal value of each harness component (retrieval moderation, Validator loop, Graph-RAG grounding) on a fixed corpus, demonstrating how to attribute gains and costs across harness layers. The Moderator security gateway instantiates the broader meta-harness concern of treating retrieved context as untrusted input requiring sanitization before passing to downstream reasoning agents. The Merkle-tree audit substrate addresses the provenance and reproducibility requirements that harness researchers increasingly flag as necessary for enterprise-grade agentic pipelines.