Skip to content

A Deterministic Control Plane for LLM Coding Agents

🕒 Published (v1): 2026-06-25 12:02 UTC · Source: Arxiv · link

Why this paper was selected

10k-repo empirical study; deterministic control plane for LLM coding agent config management

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLM coding agent configurations (CLAUDE.md, .cursorrules, etc.) are unmanaged software: exact duplicates propagate across repos, permission boundaries are rarely declared, and execution is unbounded. This paper proposes a deterministic control plane—Rel(AI)Build—that treats agent definitions as a managed supply chain with SHA-256 content addressing, tiered permissions, a phase-gated state machine, and cross-IDE compilation, all implemented in ordinary code rather than delegated to another LLM.

Problem

IDE-integrated LLM coding agents (Cursor, Claude Code, Copilot, etc.) expose broad file and shell access, yet the configuration layer that steers them is effectively unmanaged. A prevalence study of 10,008 public GitHub repos finds: 10.1% of agent config files are exact duplicates across independent repositories (fork-adjusted, SHA-256); <1% declare any permission boundaries (vs. 33% of CI/CD workflow files); 58% have a single commit; and dialects are incompatible across tools, forcing teams to re-author the same expertise multiple times. A fourth gap—unbounded execution without enforced requirement→implementation→test traceability—is not observable from static files but is motivated by empirical evidence that LLM debugging effectiveness decays sharply within 2–3 self-correction attempts.

Method

Rel(AI)Build is a Node.js control plane that operates in three planes sitting above (not replacing) existing harnesses:

  1. Authoring & Distribution Plane: A canonical registry of five resource types (agents, skills, knowledge shards, profiles, workflows). Every resource is content-addressed with SHA-256; installs recompute the hash and abort on mismatch. A per-project HMAC-SHA256-stamped lockfile detects accidental/automated corruption. An append-only, hash-chained JSONL audit log enables tamper detection: each record stores sha256(serialize(rest))[:16] for retrospective verification.

  2. Compilation Plane: A single canonical Markdown+YAML definition is compiled to seven IDE-native targets (Cursor, Claude Code, Copilot, Codex, Windsurf, OpenCode, Kiro). Static content is ordered first to maximize prefix-cache reuse. A Jaccard-similarity tokenisation-drift detector flags cosmetic prompt edits against a frozen baseline.

  3. Runtime-Governance Plane: A deterministic phase-state machine (encoded in phase-state.json) enforces ordering, mandatory delegation, and requirement→file→test traceability (trace.json). Tiered permissions (readonly / scribe / operations / specialist / orchestrator) are enforced at install/transform time as compile-time errors, not runtime exceptions. An attack-derived regex blocklist (mapped to incidents: Codecov 2021, ua-parser-js 2021, xz-utils CVE-2024-3094, etc.) blocks dangerous shell patterns across all tiers via IDE hooks (PreToolUse, beforeShellExecution).

Directory-traversal is blocked by normalizing paths and rejecting any residual .. segments. Agent-level hard constraints are embedded in system prompts (e.g., git-ops agents cannot write to .git/hooks/).

Key Contributions

  • Supply-chain architecture for agent definitions: SHA-256 content addressing, HMAC-stamped lockfiles, hash-chained audit logs, install-time integrity gates, and an agent-config SBOM mapped to NTIA minimum elements / SLSA Build Level 2.
  • Deterministic pre-execution guardrail system: tier-based permission enforcement as compile-time errors; attack-derived command/path blocklist deployed as IDE runtime hooks.
  • Tokenisation-normalisation pipeline and drift detection: Jaccard similarity measurement of prompt formatting drift as an operational risk signal.
  • Deterministic phase-gated lifecycle: a state machine enforcing process invariants (ordering, mandatory delegation, requirement→file→test traceability) that blocks execution on violation.
  • Define-once / compile-to-many transformer: a single canonical definition compiled to seven IDE targets, preserving governance properties across dialects.
  • Prevalence study: empirical characterization of governance gaps across 10,008 public repos (6,145 agent config files).

Results

  • Duplicate propagation: 10.1% of tracked agent config paths are exact duplicates across independent repositories (fork-adjusted, SHA-256); 75.5% of clone pairs cross organizational boundaries.
  • Shallow revision depth: 58% of agent config files have a single commit; age-normalized rate of 0.4 commits/month vs. 0.6 commits/month for CI/CD workflows in the same repos.
  • Permission declaration: <1% of agent config files declare permission boundaries (n=31 true positives on a fragile parser) vs. 33% of GitHub Actions workflows.
  • Credential exposure: 3.18% of tracked paths contain credential-pattern hits.
  • Dialect fragmentation: six distinct config dialects observed in corpus.
  • Conformance tests: injected violations (N=10/15/20 per mechanism) confirmed each mechanism enforces its stated invariant; no developer outcome study yet conducted.

Limitations

  • No controlled developer study: productivity, correctness, and security outcome superiority over bare harnesses are not claimed and remain future work.
  • HMAC lockfile provides detection of accidental corruption, not cryptographic authentication against an adversarial process running as the same user (who can read the local key and forge a stamp); Sigstore-grade signing is an acknowledged upgrade path not yet implemented.
  • Requirement→file→test traceability depends on cooperative agent invocation of trace-update; it is audited post-hoc, not enforced pre-execution.
  • Drift detection via Jaccard similarity is a detection mechanism, not a validated behavioral predictor of downstream model behavior changes.
  • Prevalence proxy metrics are age-confounded (commit depth) and parser-fragile (permission declarations); the unbounded-execution gap (Gap 2) is not measurable from static config files.
  • The blocklist vignette (Appendix D DRIFT) is N=1 with explicit disclaimers.
  • Reference implementation is in Node.js; portability to other runtimes is unaddressed.

Relevance to Agentic AI / LLM Agents

This paper directly addresses a governance blind spot in deployed LLM agent infrastructure: the configuration layer that steers agents with write/shell access is treated as opaque natural language rather than managed software. The central thesis—that a governance control plane must be deterministic code, not another LLM—is a principled counterpoint to the trend of using LLMs to orchestrate LLMs, and is practically relevant to anyone deploying coding agents at scale or in regulated environments. The phase-gated state machine and permission tier model operationalize concepts from AI safety (bounded autonomy, human oversight, proportional escalation) at the infrastructure level rather than the model level. The empirical finding that 75.5% of cross-org config duplicates propagate without provenance establishes a concrete supply-chain risk specific to agentic deployments that prior agent safety frameworks have not quantified.