Skip to content

Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation

🕒 Published (v1): 2026-07-13 09:07 UTC · Source: Arxiv · link

Why this paper was selected

Self-evolving agent OS that patches its own exploit kernel at runtime; novel self-improvement paradigm

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Mako introduces the Self-Evolving Agentic Operating System (SE-AOS), a new agent class where exploit capability is a mutable, versioned kernel that grows at runtime via a closed diagnose→synthesize→validate→hot-load loop. On the XBOW-104 benchmark (104 CTF-style web apps, 26 vulnerability classes), Mako achieves 104/104 (100%) coverage under a fabrication-proof verification regime. The paper's central empirical claim is a "law of autonomous exploitation": capability discovery and orchestration, not model reasoning, is the binding constraint at maturity.

Problem

Conventional tool-using agents freeze their capability set at deployment; if a required exploit primitive is absent, the agent fails and cannot recover. Additionally, prior autonomous exploitation systems lacked rigorous, fabrication-proof evaluation—results depended on model self-reports rather than verified extraction of a secret value from a live target.

Method

SE-AOS operates two coupled loops. The online exploitation loop runs a bounded perceive→reason→act cycle (MAX_TURNS=100; XBOW runs used 25): the reasoner selects one capability per turn as a typed "syscall" into the privileged capability kernel, and success is decided only when the live target emits a freshly-planted 128-bit flag in a genuine tool response.

The capability-evolution loop formalises runtime self-improvement. Let \(U\) be the universe of realisable exploit capabilities and \(T_t \subseteq U\) the kernel at step \(t\). The coverage functional is: $\(C(T) = \mathbb{E}_{e \sim \mathcal{E}}\bigl[\mathrm{solve}(e, T)\bigr]\)$ The evolution operator \(\Phi\) maps a failure trace \(\tau_t\) and current kernel to a candidate primitive, admitted only if it passes the fabrication-proof gate \(V\): $\(c_{\mathrm{new}} = \Phi(\tau_t, T_t), \quad T_{t+1} = T_t \cup \{c_{\mathrm{new}} : V(c_{\mathrm{new}}) = 1\}\)$ Because capabilities are additive and selection is incumbent-preserving, \(C(T_{t+1}) \geq C(T_t)\); the sequence is nondecreasing and converges to a fixed point \(T^\star\) with \(C(T^\star) = 1\).

Tool-selection law: if a solving capability \(c^\star \in T\) exists with per-turn selection probability \(p\), \(\mathbb{E}[\text{turns}] \approx 1/p\). Discoverability engineering (keyword-rich descriptions, umbrella tool chaining) drives \(p \to 1\), collapsing expected turns to 1. This accounts for the observed L3 inversion.

Models: Gemini 2.5 Flash (primary); one-way escalation to Gemini 3.1 Pro Preview after consuming 70% of turn budget without a confirmed finding (\(\rho = 0.7\)). Tool arsenal: ~180 registered tools; ~50 newly built or materially enhanced during the campaign. A parallel platform-level \(\Phi\) loop improves the agent's own detection rules and prompts under a fitness regression gate, without human per-cycle review.

Key Contributions

  • SE-AOS architectural class: capability as a mutable, versioned, hot-loadable kernel rather than a frozen deployment artifact.
  • Formal capability-evolution operator \(\Phi\) with a monotone coverage proof and submodular marginal-gain analysis yielding a \((1-1/e)\) greedy prioritisation heuristic.
  • Fabrication-proof evaluation regime: fresh 128-bit flag injected per build via Docker build-arg; success iff the flag literally appears in a real tool response, never a model self-report; \(\Pr[\text{false positive}] \leq 2^{-128}\).
  • 104/104 full-suite coverage of XBOW-104 across all 26 vulnerability classes and all three difficulty tiers.
  • Law of autonomous exploitation: capability (discoverability and orchestration), not model reasoning, is empirically the scarce resource at maturity.
  • Turn-economics and cost analysis by category and difficulty, including the L3 inversion (hardest tier solved fastest once purpose-built tool exists).
  • Dual-use responsible-disclosure posture: science published, operational artefacts withheld.

Results

  • Coverage: 104/104 (100%) on XBOW-104 — vs. KinoSec 103/104 (99.0%, self-reported, no disclosed verification methodology), BoxPwnr 101/104 (97.1%), Shannon Lite 100/104 (96.2%, white-box), Strix v0.4.0 100/104 (96%), XBOW proprietary ~85%, MAPTA 80/104 (76.9%), 5-person human team 91/104 (87.5%), 20-year expert (40h) ~85%.
  • Turn economics: median 7 turns/solve, mean 10.5, min 2, max 40 (\(n=104\)); 17 benchmarks solved in 2 turns (one exploit call + report).
  • L3 inversion: L3 (hardest, 8 benchmarks) median 2.0 turns vs. L1 median 7.5 and L2 median 7.0.
  • Compute cost: \(478.99 total (\)306.44 Gemini 2.5 Flash + \(172.55 Gemini 3.1 Pro); ≈\)4.61 per benchmark at official list pricing.
  • Token volume: ≈1.03B tokens end-to-end (~95% input); input-dominated by re-sending ~180 tool schemas + running memory each turn.
  • Discoverability evidence: XBEN-023 moved from 18-turn failure to 2-turn solve solely by improving tool description—no new exploit logic.

Limitations

  • Operational payloads, proof-of-concept code, per-target exploit chains, and tool source are withheld; independent reproduction is not possible.
  • Per-turn model was not logged; the exact proportion of benchmarks that triggered the Gemini 3.1 Pro escalation is unconfirmed.
  • ~42/104 upstream fixtures required infrastructure repair (EOL package archives, ARM64 incompatibilities, missing service ports); while the authors argue changes are git-revertible and do not touch application logic, this cannot be independently audited without the tool source.
  • The submodular \((1-1/e)\) bound and the no-regression proposition assume an incumbent-preserving, deterministic selection policy; the actual \(\pi_\theta\) is stochastic, so guarantees hold in expectation only, not pointwise.
  • Results are on a single, finite, CTF-style benchmark suite; generalisation to novel, unseen production targets is undemonstrated.
  • The platform-level \(\Phi\) evolution loop cannot rewrite its own control plane (by design for safety), limiting the depth of self-modification.
  • KinoSec comparison in Table II carries significant caveats (no independent verification, no run data disclosed), limiting the competitive picture.

Relevance to Agentic AI / LLM Agents

Mako directly advances the empirical understanding of where capability bottlenecks live in tool-using LLM agents: the paper provides rigorous evidence that tool discoverability and orchestration dominate model reasoning ability as the scarce resource once the capability kernel reaches a sufficient level of coverage, a finding with broad implications beyond security for any agentic system relying on a large tool library. The SE-AOS architecture — a versioned, self-extending capability kernel with a verification-gated evolution loop — is a concrete implementation of runtime self-improvement that is distinct from prompt-based or weight-based approaches and may generalise to other long-horizon agentic domains. The fabrication-proof evaluation regime addresses a fundamental open problem in agentic benchmarking: how to prevent LLM agents from passing evaluations via self-report or memorisation. The cost result (~$4.61 per full web-exploitation solve at commodity rates) concretises the economic accessibility of mature agentic pipelines.