Architectural Design Decisions in AI Agent Harnesses¶

🕒 Published (v1): 2026-04-20 10:39 UTC · Source: Arxiv · link

Why this paper was selected

Systematic survey of architectural design decisions specific to AI agent harnesses

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

An empirical study of 70 publicly available AI agent-system projects examines the non-LLM engineering infrastructure (termed "agent harnesses") through five design dimensions: subagent architecture, context management, tool systems, safety mechanisms, and orchestration. The study identifies recurring architectural patterns, documents co-occurrence structure among design decisions, and refutes several assumed simplifications (e.g., "language determines architecture"). The result is an evidence-based taxonomy and pattern catalog for agent-harness engineering.

Problem¶

Prior work on LLM-based agents addresses capabilities (reasoning, tool use, multi-agent coordination) but leaves the surrounding non-LLM engineering infrastructure—tool mediation, context handling, delegation, execution boundaries, safety governance—comparatively undertheorized. There is no cross-project comparative account of which infrastructural decisions recur, which co-occur, and how different product positions map to architectural commitments. This leaves framework developers without empirical guidance and practitioners without a basis for principled framework selection.

Method¶

Protocol-guided, source-grounded qualitative empirical study of a 70-project corpus (67 open-source repositories + 3 public-evidence comparison cases; corpus frozen 2026-03-23). Projects were screened to include systems with meaningful surrounding infrastructure (>~500 LOC or equivalent footprint, at least one inspectable capability beyond a single prompt-to-API path); pure prompt collections, thin wrappers, benchmarks, and demos were excluded. Each project was coded along five focal design-decision dimensions (Table 2), with decisions recorded as presence/absence observations or categorical labels. RQ1 uses descriptive summarization of option spectra; RQ2 uses cross-project co-occurrence analysis; RQ3 uses pattern synthesis across complexity and positioning strata. Large projects used sampled code reading with declared coverage; projects lacking full source used public technical materials (papers, architecture docs, blog posts) with explicit evidence trails.

Key Contributions¶

Empirical characterization: Five recurring design dimensions—subagent architecture, context management, tool systems, safety mechanisms, orchestration—identified from source evidence and established as a coherent analytical basis via a transparent, protocol-guided procedure.
Co-occurrence and non-co-occurrence analysis: Demonstrates the design space is structured by decision bundles, not isolated feature choices; refutes three common simplifications ("language determines architecture," "use case fixes complexity," "capability growth implies governance maturity").
Pattern synthesis: Five recurring architectural patterns (lightweight tools, balanced CLI frameworks, multi-agent orchestrators, enterprise systems, scenario-verticalized projects) that turn cross-project variation into a reusable comparative framework.
Auditable research package: Documented corpus boundary, search/screening protocol, and operational codebook in appendices, enabling later studies to audit, refine, or extend the comparison.

Results¶

Context management: Corpus favors file-persistent, hybrid, and hierarchical context strategies across projects.
Tool systems: Registry-oriented tool systems remain dominant; MCP- and plugin-oriented extension mechanisms are emerging but not yet prevalent.
Safety: Intermediate execution isolation is common; high-assurance auditing is rare.
Co-occurrences observed:
Deeper coordination pairs with more explicit context services.
Stronger execution environments pair with more structured governance.
Formalized tool-registration boundaries pair with broader ecosystem ambitions.
Non-co-occurrences (refuted assumptions):
Implementation language does not determine architectural choices.
Use-case domain does not fix complexity level.
Expanded capability does not imply governance maturity.
Patterns synthesized: 5 archetypal architectural configurations spanning lightweight single-tool agents through enterprise-scale multi-agent orchestrators.

Limitations¶

Cross-sectional design (single snapshot per project); cannot capture architectural evolution or temporal dynamics.
Not a PRISMA-style exhaustive census; corpus reflects "analytic coverage" with snowball expansion, so selection bias cannot be fully excluded.
Three projects lacked full source visibility and relied on public technical materials, introducing potential inference gaps.
Five focal dimensions do not exhaust all possible infrastructural concerns; the authors acknowledge a broader evidence base was reduced to these five.
Qualitative coding introduces subjectivity despite the protocol; inter-rater reliability statistics are not reported in the provided text.
Corpus size (70) may limit statistical power for co-occurrence claims, especially for rare feature combinations.

Relevance to Harnesses / Meta-Harnesses¶

This paper is the most directly relevant study possible for researchers tracking this topic: it treats the "agent harness" as the primary unit of analysis, provides the first systematic cross-project taxonomy of harness design decisions, and names five architectural dimensions (subagent architecture, context management, tool systems, safety mechanisms, orchestration) that constitute a shared vocabulary for the field. The co-occurrence findings—particularly that coordination depth pairs with context explicitness, and that tool-registration formalism pairs with ecosystem ambition—offer structural hypotheses about how meta-harness capabilities compound. The five synthesized patterns (lightweight, CLI-balanced, multi-agent orchestrator, enterprise, scenario-verticalized) directly map onto the architectural space that meta-harness designers navigate when choosing what infrastructure to wrap or reuse.