Model-Native Computing Architecture: Envisioning Future System Architecture Through the Lens of Computer Architecture¶

🕒 Published (v1): 2026-05-29 19:20 UTC · Source: Arxiv · link

Why this paper was selected

Frames LLM agent scheduling as classical computer-architecture problem; novel systems lens

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This visionary survey maps eight decades of computer-architecture concepts onto the emerging LLM system stack, identifying a fundamental gap: independently developed techniques (prefix caching, chunked prefill, sub-agents, MCP, persistent memory) lack a unifying abstraction layer. To fill this gap, the authors propose the Intelligent Computing Architecture (ICA)—a six-layer framework with a dual-plane split (probabilistic execution / deterministic control) and three Amdahl-style design heuristics. No new empirical results are presented; this is a conceptual and taxonomic contribution.

Problem¶

LLM systems now exhibit the same layered complexity as classical computing stacks, yet lack a unified architectural language. Existing OS-analogy efforts (AIOS, MemGPT, ArbiterOS, MemOS) each address one sublayer and contradict each other on the core metaphor ("Is the LLM a CPU or an OS?"), preventing principled cross-layer composition and making each new system a from-scratch rebuild. Additionally, nearly all prior frameworks treat the human as a boundary condition rather than a core loop participant, despite empirical evidence that agents handle ~60% of engineering work but receive full autonomous delegation for only 0–20% of tasks.

Method¶

The paper proceeds in three steps:

Analogy framework: Classical computing layers (ISA → microarchitecture → cache → virtual memory → OS → distributed systems) are mapped one-to-one onto model-native counterparts (model weights → inference serving → KV cache → context/RAG → agent runtime → multi-agent coordination), with explicit enumeration of where the analogy holds and where it breaks.
Dual-plane architecture: The persistent "CPU vs. OS" metaphor conflict is resolved by partitioning the stack into two orthogonal planes:
Probabilistic execution plane: model inference and generation (what can be computed)
Deterministic control plane: scheduling, permissions, and orchestration (what should be computed) Every ICA layer is a graded crossover through both planes.
Intelligent Computing Architecture (ICA): Six functional layers with inter-layer interface contracts and six design axioms, mirroring the classical six-layer stack. Three Amdahl-style back-of-envelope heuristics are proposed:
Semantic Locality: analogous to cache locality—frequently reused context prefixes should be cached.
Context Budget: analogous to Amdahl's serial fraction—the fraction of tokens unavoidably in the working context caps total throughput.
Agent Speedup: analogous to parallel speedup—the fraction of tasks safely delegatable to sub-agents bounds end-to-end latency reduction. Formally: $$S = \frac{1}{(1-f) + f/p}$$ transplanted to the agent scheduling regime.

Parameter ranges for the heuristics are illustrated with published system data (vLLM, DistServe, Mooncake, etc.) but the authors explicitly disclaim these as validated scaling laws.

Key Contributions¶

Analogy framework mapping classical computer-architecture abstractions to the model-native stack, with explicit delineation of breakdown points.
Dual-plane architecture resolving the CPU/OS metaphor conflict via probabilistic execution plane + deterministic control plane.
ICA: six-layer framework with defined inter-layer interfaces and six design axioms.
Three Amdahl-style design heuristics (Semantic Locality, Context Budget, Agent Speedup) with parameter ranges from published data; framed explicitly as back-of-envelope intuition, not validated laws.
Survey of open-source implementations (vLLM, MemGPT, AIOS, MemOS, Claude Code, OpenHands, AutoGen, MetaGPT) and a research roadmap.

Results¶

This paper presents no new experimental results. Quantitative claims are illustrative parameter recoveries from published third-party data: - Engineers at Rakuten ran Claude Code autonomously over a 12.5-million-line codebase for hours. - An Augment Code customer compressed a 4–8 month project to two weeks. - AI handles ~60% of engineering work but receives full autonomous delegation for only 0–20% of tasks. - Developers accept ~30% of AI code suggestions on average. - Amdahl's Law example: $f=0.8$, $p=4$ → $S=2.5\times$ (vs. ideal $4\times$); ceiling at $p\to\infty$ is $5\times$.

Limitations¶

No experimental validation; all heuristics are back-of-envelope models with parameters reverse-solved from existing data rather than derived from first principles or confirmed by forward prediction.
The analogy framework is explicitly bounded—the paper enumerates breakdowns but does not provide a formal criterion for when the analogy fails.
The collaboration paradox (high use, low delegation) is observed and named but not resolved; the ICA does not specify a mechanism for safely expanding the delegatable fraction.
The human-in-the-loop is incorporated as a system component conceptually, but no concrete interface contract is specified for the human layer.
Coverage of non-Transformer architectures (Mamba, SSMs) is brief; their impact on the KV-cache/virtual-memory analogy is not worked through.

Relevance to Harnesses / Meta-Harnesses¶

The ICA's Layer 4–5 (Interface / Orchestration) and Layer 5–6 (Multi-Agent / App) directly describe what a harness or meta-harness does: task decomposition, tool dispatch, sub-agent scheduling, sandbox isolation, permission control, and result aggregation. The dual-plane split—probabilistic execution for model calls, deterministic control for orchestration logic—maps precisely onto the architectural tension harness designers already navigate: when to trust the model's output and when to enforce hard constraints programmatically. The three design heuristics give harness architects computable intuitions for cache reuse policy (Semantic Locality), context budget allocation (Context Budget), and sub-agent parallelism limits (Agent Speedup). The framing of MCP as a vertical I/O bus and A2A as a horizontal interconnect clarifies the protocol layer that harnesses must either implement or sit atop.