Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production¶

🕒 Published (v1): 2026-07-08 06:27 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Progressive crystallization is a lifecycle that converts successful LLM agent execution traces into progressively cheaper, deterministic workflows—moving from fully stochastic agent runs (Type 3) through hybrid (Type 2) to zero-token deterministic playbooks (Type 1). Promotion is evidence-gated and demotion is automatic on regression. In a production cloud-network AIOps system, this reduced per-incident agent cost by >70% while incident volume doubled, with 45% of executions served deterministically after eight months.

Problem¶

LLM agents deployed for IT operations are permanent cost centers: every execution re-invokes full model inference even for incidents the agent has already solved repeatedly. Successful agent investigations are discarded—the next occurrence re-discovers the solution from scratch at the same token cost, non-deterministically. Traditional workflow engines, fine-tuning, and RPA capture neither the reasoning nor the data flow and cannot handle novel scenarios.

Method¶

The paper defines a three-type execution taxonomy and a promotion/demotion lifecycle:

Type 3 (Agent-orchestrated): Free agent investigation within bounded scope; read ops autonomous, write ops behind HITL gates; ~10k–50k tokens/run.
Type 2 (Hybrid): Fixed step structure; LLM invoked only for interpretation/classification at specific stages; ~1k–5k tokens/run.
Type 1 (Deterministic): Pre-coded logic with typed API calls; zero LLM tokens; millisecond latency.

The four lifecycle stages are: 1. Discovery: Novel incident triggers a Type 3 run with full trace recording. 2. Capture: Successful trace is parsed into a reusable Type 3 template by extracting ordered tool calls, branch conditions, input/output schemas, a DAG of tool dependencies, and parameterizing instance-specific values. 3. Promotion to hybrid (Type 3→2): After ≥10 successful runs with zero safety violations, ≥90% action-sequence agreement, and passing auto-generated acceptance tests, LLM-consistent classification steps are replaced by deterministic rules; variable-but-stable reasoning steps become scoped single-purpose prompts. 4. Promotion to deterministic (Type 2→1): After ≥50 hybrid runs with ≥99% LLM classification consistency and full regression suite passing without the LLM, remaining LLM steps whose output is finite or rule-expressible are replaced with deterministic equivalents.

A circuit-breaker demotes any promoted playbook on execution failure, safety violation, or acceptance-test regression, then allows re-promotion after clean runs.

Key Contributions¶

A three-point execution-type taxonomy (Type 3/2/1) placing operational workflows on a stochastic-to-deterministic spectrum.
An evidence-based promotion/demotion lifecycle with explicit, configurable promotion criteria (Table II).
A trace-extraction algorithm that converts agent execution traces into parameterized, DAG-structured playbook templates via process mining techniques.
A safety-monotonicity argument: reproducibility rises from ~50% (Type 3) to 100% (Type 1); blast-radius control moves from runtime HITL gates to statically verifiable deterministic logic; compliance becomes built-in rather than conditional.
An economic model showing platform inference cost decreases over time as volume grows, because Type 1 executions amortize the agent's discovery cost across all future runs.
A platform-maturity metric: the Type 1:2:3 execution ratio.

Results¶

Over eight months in a production cloud-network AIOps system handling tens of thousands of incidents/month:
Type 1 (deterministic) execution share rose from 0% to ~45%; ~30% hybrid, ~25% agent-orchestrated.
Per-incident agent cost fell by >70% while incident volume roughly doubled.
90% of common incident categories resolved autonomously.
Mean time to resolution fell from hours to minutes.
False-positive remediation rate remained under 5% with no customer-visible impact.
Safety properties improved monotonically across all promotions.

No external baselines are compared against; results are production observations from a single deployment.

Limitations¶

Results are from a single organization and operational domain; specific thresholds and ratios require re-derivation elsewhere.
Benefit is diminished in environments dominated by genuinely novel, one-off incidents where most executions remain Type 3.
Economic figures are platform-level observations, not a controlled comparison.
Automatic promotion depends on acceptance-test quality; under-observed patterns can be promoted prematurely.
Extraction quality is limited by trace richness—sparse or poorly structured logging constrains what can be crystallized (a known dependency in process mining).
Eight-month window reflects one maturity trajectory, not a steady-state measure.

Relevance to Harnesses / Meta-Harnesses¶

Progressive crystallization is directly relevant to meta-harness design: it formalizes the idea that a harness is not a fixed execution engine but a lifecycle manager that routes work to different execution tiers based on accumulated evidence. The trace-extraction and template-promotion mechanism is essentially an automated harness self-optimization loop—the meta-harness observes agent runs, crystallizes proven paths into cheaper sub-harnesses (Type 2/1 playbooks), and falls back to the full agent harness for novel inputs. The demotion circuit-breaker maps directly to the regression-detection and fallback logic that production meta-harnesses require. The Type 1:2:3 maturity ratio offers a concrete, operational metric for evaluating harness efficiency over time.