The Semi-Executable Stack: Agentic Software Engineering and the Expanding Scope of SE

🕒 Published (v1): 2026-04-16 18:36 UTC · Source: Arxiv · link

Why this paper was selected

Agentic harnesses expand scope of SE; conceptual framing for the field

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This conceptual keynote companion argues that agentic software engineering expands rather than contracts the scope of SE, because LLM-based tool-using agentic harnesses cause organizations to engineer a broader class of "semi-executable artifacts"—prompts, workflows, evaluation harnesses, and organizational routines—that shape system behavior but resist classical deterministic execution. The Semi-Executable Stack is introduced as a six-ring diagnostic reference model spanning from executable code (ring 1) through instructional artifacts, orchestrated execution, control systems, operating logic, and societal/institutional fit (ring 6). The paper is diagnostic and agenda-setting, not empirical.

Problem

SE discourse defaults to a "shrinking field" narrative where improving LLM agents displace engineers from coding tasks. This framing lacks an analytic vocabulary for the artifacts that actually govern agentic system behavior in practice—prompts, evaluation harnesses, orchestration logic, escalation rules, and organizational routines—none of which are classical code yet all of which directly constrain runtime behavior. No prior reference model organizes these artifacts along a shared executability spectrum or makes clear where engineering leverage and bottlenecks reside.

Method

The paper introduces the Semi-Executable Stack, a six-ring reference model organized along a spectrum defined jointly by specification completeness and execution determinism. Rings are not lifecycle phases or runtime layers; they denote regions where enactment depends increasingly on probabilistic model interpretation or human judgment rather than deterministic execution:

1. Executable artifacts — code, tests, schemas, CI configs
2. Instructional artifacts — prompts, NL specifications, exemplars
3. Orchestrated execution — tool use, retrieval, agent workflows, multi-agent protocols, human-agent loops
4. Control systems — guardrails, monitoring, evaluation harnesses, policy layers, escalation rules
5. Operating logic — decision preparation, coordination routines, delegation structures, knowledge reuse
6. Societal and institutional fit — regulatory compliance (EU AI Act), cross-org legitimacy, public-sector adoption

A semi-executable artifact is formally defined as one whose enactment depends on interpretation by humans, probabilistic models, or both, rather than fully deterministic machine execution alone. The model is applied diagnostically via five questions (primary ring, interpretation locus, evaluation criteria location, human unit of analysis, adjacent-ring dependencies) and demonstrated through three worked cases (SPAPI-Tester, GoNoGo, The Cybernetic Teammate). A preserve-versus-purify heuristic guides decisions on which legacy SE processes to retain versus redesign.

Key Contributions

• Formal definition of semi-executable artifact as a distinct engineering object class
• Six-ring Semi-Executable Stack diagnostic reference model
• Diagnostic usage table (Table 1) mapping engineering questions to ring-assignment procedures
• Three worked cases contrasting contributions at rings 2–3, 5 (with 3–4 support), and 5
• Five reframing moves treating common objections ("SE is shrinking") as engineering targets
• Preserve-versus-purify heuristic for legacy SE process decisions
• Behavioral software engineering (BSE) identified as a cross-cutting lens for tracking human-unit-of-analysis effects

Results

This is a conceptual paper; no empirical benchmarks are reported. Diagnostic validity is argued through three illustrative cases:

• SPAPI-Tester (rings 2–3): automated API-to-test pipeline across 193 production APIs surfacing 23 failures (22 confirmed); contributes a ring-2/3 recipe for prompt+workflow engineering without organizational redesign
• GoNoGo (rings 5 with 3–4): re-engineers a release-decision support routine at the operating-logic ring; primary contribution is coordination and information-flow redesign, not code generation
• The Cybernetic Teammate (ring 5): demonstrates AI reshaping expertise combination and work coordination beyond software-generation tasks; domain transfer to SE settings remains open

No accuracy, latency, or capability numbers specific to the stack model itself are reported.

Limitations

• Entirely conceptual/diagnostic; no empirical validation of the six-ring taxonomy or heuristic
• Not a maturity model, process framework, or evaluation rubric—prescriptive completeness is explicitly out of scope
• Cases are illustrative and drawn partly from a single automotive setting; generalizability is asserted, not tested
• Ring 6 (societal and institutional fit) is acknowledged as not directly engineered in the same sense as inner rings—it is treated as an external constraint, which may understate engineering agency at that level
• The preserve-versus-purify heuristic is stated but not operationalized with decision criteria

Relevance to Harnesses / Meta-Harnesses

The paper explicitly situates evaluation harnesses as a first-class engineering object at ring 4 (Control systems), alongside guardrails, monitoring, and policy layers—framing them not as test scaffolding but as engineered artifacts that gate releases and maintain operational correctness. The abstract opens by defining current agentic AI systems as "LLMs wrapped in tool-using agentic harnesses," making the harness concept foundational to the paper's entire argument. For researchers tracking harnesses and meta-harnesses, the Semi-Executable Stack provides a vocabulary to locate where a given harness contribution sits (ring 3 for orchestration harnesses, ring 4 for evaluation/control harnesses, ring 5 for operating-logic harnesses) and to diagnose which adjacent rings a harness design depends on for deployment viability. The observation that prompts, skills, tool permissions, and reusable workflow structures in systems like Claude Code increasingly encode first-class harness logic (ring 2–3) directly foregrounds meta-harness design as a SE concern.