Skip to content

KISS Sorcar: A Stupidly-Simple General-Purpose and Software Engineering AI Assistant

🕒 Published (v1): 2026-04-26 17:59 UTC · Source: Arxiv · link

Why this paper was selected

Minimal SE assistant addressing finite context windows and multi-turn session gaps

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

KISS Sorcar is a five-layer AI agent framework (~1,850 lines of Python) and VS Code extension for software engineering, built on a strict KISS (Keep It Simple, Stupid) design philosophy. Each layer adds exactly one concern—budget-tracked ReAct execution, context-window continuation, coding/browser tools, multi-turn chat persistence, and git worktree isolation. On Terminal Bench 2.0 it achieves 62.2% pass rate (Claude Opus 4.6), outperforming Claude Code (58%) and Cursor Composer 2 (61.7%).

Problem

Existing LLM-based coding agents fail in practice due to finite context windows, single errors derailing entire sessions, agents stuck in dead ends, low-quality "AI slop" output, and changes that are hard to review or revert. There is no widely available minimal-footprint framework that cleanly separates these concerns into composable layers.

Method

The KISS Agent Framework implements a strict five-layer inheritance chain:

  1. KISS Agent — ReAct loop with native function calling, per-agent and global cross-agent budget/token/cost tracking, error resilience with retryable vs. non-retryable error detection, and stateless-per-run execution.
  2. Relentless Agent — wraps KISS Agent in a continuation loop; when a sub-session exhausts its context or step budget, it forces a structured chronological summary (with code snippets) via finish(is_continue=True), spawns a separate summarizer agent if the session crashes, and resumes with accumulated progress.
  3. Sorcar Agent — adds Bash/Edit/Write/Read coding tools, Playwright browser automation, optional parallel sub-agent thread pool, Docker-aware tool variants, and an ask-user-question interrupt.
  4. Chat Sorcar Agent — persists multi-turn sessions to a local database keyed by chat ID; prepends prior task/result history to each new prompt.
  5. Worktree Sorcar Agent — creates a git branch and worktree per task; copies dirty working-tree state as a baseline commit; cherry-picks only agent changes on merge; uses per-repo file locks for concurrency safety; stores all state in git itself for crash recovery.

A structured system prompt encodes engineering discipline: explicit tool-use rules (Write vs. Edit, timeout guidance, chunked file reads), forced self-continuation near the step budget, and anti-slop directives ("NO AI SLOP", "BE RELENTLESS", "BE CALM").

Key Contributions

  • Five-layer composable agent hierarchy (~1,850 LOC total) where each layer is independently small and single-concern.
  • Relentless continuation protocol: forced chronological summarization across sub-sessions without context-compaction techniques; crash-recovery via a separate summarizer agent reading the full trajectory file.
  • Git worktree isolation layer with dirty-state preservation, concurrency locking, and crash recovery entirely via git metadata.
  • Structured system prompt encoding a reproducible engineering discipline (tool rules, budget-step thresholds, extended reasoning activation).
  • Self-hosting validation: the entire codebase was built using itself over 4.5 months, providing continuous integration-style stress testing.
  • Benchmark audit finding: identifies cheating in top Terminal Bench 2.0 submissions (verifier leakage, hardcoded answers) and process-isolation failures in 45 solutions across 13 benchmarks.

Results

  • Overall pass rate: 62.2% (277/445 trials) on Terminal Bench 2.0 (89 tasks, 5 trials each, Claude Opus 4.6).
  • pass@any (≥1/5 pass): 78.7% (70/89 tasks).
  • pass@all (all 5/5 pass): 43.8% (39/89 tasks).
  • Always-fail: 19 tasks; Always-pass: 39 tasks; Mixed: 31 tasks.
  • Median cost/trial: $0.45; Mean cost/trial: $0.90.
  • Median duration/trial: 202 s; Mean duration/trial: 446 s.
  • Outperforms Claude Code (58%) and Cursor Composer 2 (61.7%) on the same benchmark with the same base model (Opus 4.6).
  • Consistent failures cluster in: GUI/multimedia-dependent tasks, resource-exhausting builds, and narrow niche domains (protein assembly, OCaml GC patching).

Limitations

  • Summary-based continuation may become unwieldy for multi-day tasks; no formal evaluation of summary scaling is provided.
  • Parallel sub-agent tool is disabled by default in the VS Code IDE due to inability to coherently stream multiple outputs.
  • Hard-skips 9 tasks deemed infeasible for Opus 4.6 (still counted as failures), potentially overstating efficiency.
  • Benchmark context: KISS Sorcar scores lower than some leaderboard entries, though the paper attributes those higher scores to cheating rather than genuine capability.
  • Quality-first posture accepts high latency (mean 446 s/trial, mean $0.90/trial), which is impractical for latency-sensitive workflows.
  • No ablation isolating the contribution of individual layers vs. the system prompt.

Relevance to Harnesses / Meta-Harnesses

KISS Sorcar is a textbook case of a layered meta-harness: each agent class is itself a harness over the one below, with the outermost layer (Worktree Sorcar Agent) functioning as a task orchestration shell that spawns, isolates, and merges work from inner agents. The Relentless Agent's continuation protocol—forced summarization, trajectory-file handoff to a summarizer sub-agent, and session resume—is exactly the kind of sub-session lifecycle management that meta-harness designers must solve. The structured system prompt, acting as a declarative behavioral spec enforced across all layers, parallels the role of configuration schemas in more complex harness architectures. For researchers tracking harness design patterns, this paper provides a minimal, fully public reference implementation (~1,850 LOC) with empirical benchmark validation.