Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting¶

🕒 Published (v1): 2026-06-28 18:30 UTC · Source: Arxiv · link

Why this paper was selected

Practitioner-focused loop-design framework; reframes harness architecture around autonomous iteration

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This position paper formalizes "loop engineering" as a new discipline layer above prompt, context, and harness engineering, defining a loop specification as a bounded, reusable artifact (trigger + goal + verification + stopping rule + memory) that a human hands to an agent harness to pursue work autonomously. It introduces a five-level verification ladder, codes a corpus of 50 real loops from the public Loop Library, and derives design principles from the self-correction and reward-hacking literature.

Problem¶

The mid-2026 practitioner shift from step-by-step prompting to "designing the loop that prompts the agent" emerged through blogs and threads with no reviewable account, leaving the concept undefined, unscoped, and unsupported by evidence. Without a clear vocabulary, "loop" conflates three distinct things—control-flow loops, the internal perceive-act-observe harness cycle, and the new external loop specification—and practitioner claims cannot be examined against the scientific literature.

Method¶

The paper proceeds in three moves:

Conceptual scoping: defines the loop specification as a five-part external artifact \(\langle \text{trigger}, \text{goal}, \text{execution}, \text{verification}, \text{stopping rule} \rangle\) plus durable file-based memory, and situates it as a fourth layer above prompt → context → harness engineering.
Taxonomy: constructs an anatomy (Figure 2) and classifies loops by trigger type (manual/scheduled/event), goal type (verifiable/judged/mixed), a five-level verification ladder (L1 deterministic → L2 rule → L3 delayed field truth → L4 model-as-judge → L5 human checkpoint), architecture (solo/maker-checker/manager-helper), and named terminal states (success/no-op/blocked/stalled/exhausted).
Corpus study: hand-codes all 50 loops in the public Loop Library against the taxonomy dimensions, reporting counts and proportions without running the loops or measuring budgets.

Additionally, the paper releases sandeco-loop, an open skill implementing the loop-authoring procedure, and derives design principles and anti-patterns from cited literature on self-correction, reward hacking, and model-as-judge fragility.

Key Contributions¶

Precise definition and three-way disambiguation of "loop" (control flow vs. internal agent cycle vs. external loop specification).
Four-layer engineering progression: prompt → context → harness → loop, each subsuming the prior.
Five-level verification ladder distinguishing the autonomous zone (L1–L2), objective zone (L1–L3), and assisted flow (L4–L5).
Descriptive hand-coded analysis of the 50-loop Loop Library corpus with per-loop JSON artifacts.
Four design-principle families (define "done"; act without breaking; earn trust; sustain over time) grounded in cited literature on reward hacking, self-correction failure, and memory governance.
Enumerated anti-patterns and evaluator criteria.
Release of sandeco-loop, a public skill implementing the loop-authoring procedure.

Results¶

All figures are from the 50-loop corpus study: - 70% of loops verify in the autonomous zone (L1 or L2); 76% stay within the objective zone (L1–L3). - 50% use deterministic (L1) verification as their dominant check; ~20% use rule-based (L2). - 22% use a model-as-judge (L4) as dominant check; only 1 loop relies on a human checkpoint (L5). - Of the 11 L4-dominant loops, most pair with a maker-checker architecture rather than self-judging. - 78% of loops use manual triggers; 12% scheduled; 10% event-driven. - 78% use a solo-agent architecture; 18% maker-checker; 4% manager-helper. - 74% name their terminal states; 66% set a verifiable goal. - Only 22% use an automated trigger; only 20% call named reusable skills; only 32% develop persistent memory. - Pattern family coverage: "define done" (A) 44%, "earn trust" (C) 40%, "act without breaking" (B) 38%, "sustain/memory" (D) 32%.

No benchmark comparisons against baselines; this is a position + corpus paper, not an empirical evaluation.

Limitations¶

Position paper with a descriptive corpus study; no controlled measurements of loop performance, cost, latency, or failure rates.
Corpus is a single public catalog (Loop Library, \(n=50\)) curated by one organization; generalizability is unverified.
Hand-coding is by a single coder (the author); inter-rater reliability is not reported.
Four loops had pattern families inferred rather than read from catalog text.
The sandeco-loop skill is released but not evaluated against alternative loop-authoring procedures.
The paper explicitly defers empirical budget-controlled loop execution studies to future work.
Verification ladder levels are defined conceptually but the boundary between L2 (rule) and L3 (delayed field truth) may be ambiguous in practice.

Relevance to Harnesses / Meta-Harnesses¶

This paper directly theorizes the layer above harness engineering: it treats harnesses (Claude Code, Codex) as the runtime engine and defines loop specifications as the meta-level pilot written to orchestrate them, making it a foundational conceptual contribution to the harness/meta-harness design space. The five-part anatomy—trigger, goal, verification, stopping rule, memory—is essentially a schema for authoring meta-harnesses, and the five-level verification ladder provides a principled vocabulary for evaluating autonomous harness behavior that is currently absent from the literature. The corpus evidence that automated triggering, skill reuse, and persistent memory are underdeveloped directly identifies the engineering gaps where meta-harness work has the most leverage. The maker-checker architectural pattern and the explicit prohibition against a loop self-approving its own output are directly actionable harness-design principles.