Skip to content

BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledge

🕒 Published (v1): 2026-05-15 10:09 UTC · Source: Arxiv · link

Why this paper was selected

BootstrapAgent: distills costly repository setup knowledge into reusable agent memory

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

BootstrapAgent is a multi-agent framework that converts the costly, one-off process of setting up an unfamiliar repository into a persistent, verifiable .bootstrap contract reusable by future agents. It combines evidence-driven discovery, structured planning, Docker-based verification, and trace-driven delta repair to achieve a 92.9% bootstrap success rate, reducing downstream agent token usage by 25.9% and build time by 22.3%.

Problem

Repository bootstrapping—getting a freshly cloned repo into a working development state—is a high-cost, trial-and-error process that consumes substantial tokens and time for every code agent that encounters the same repository. The discovered knowledge (resolved dependencies, working repair strategies) is ephemeral: it dies with the conversation and cannot be reused, forcing subsequent agents to repeat the same expensive exploration. Files like AGENTS.md/CLAUDE.md encode coding guidance but do not address environment setup or failure recovery.

Method

BootstrapAgent treats bootstrapping as producing a reusable startup knowledge artifact rather than a one-time deployment task. The pipeline has five stages:

  1. Repository Discovery: A Discoverer Agent scans READMEs, package metadata, lockfiles, Makefiles, build configs, and CI workflows, emitting a structured DiscoveryReport and CIEvidenceReport (with non-reproducible CI features flagged).
  2. Bootstrap Plan Generation: A Planner Agent converts the evidence into a BootstrapPlan—a schema-constrained document fixing setup phases, verification goals, and evidence links. The plan acts as a stable task anchor across repair rounds.
  3. Contract Generation: A Generator Agent materializes the plan into a .bootstrap directory containing setup.sh, doctor.sh, verify.sh, command metadata, an evidence map, compressed agent context, and a failure playbook. The contract is formally \(C = (I, D, M, S, H)\) where \(I\) = setup commands, \(D\) = diagnostic checks, \(M\) = mandatory minimal verification, \(S\) = optional strongest locally-reproducible check, and \(H\) = accumulated repair knowledge.
  4. Containerized Verification: A Docker Verifier replays the contract in a clean container, producing an execution trace \(\tau_t = V(R, C_t; B)\) with exit codes, outputs, and failure locations.
  5. Trace-Driven Repair: The Generator Agent proposes a structured delta \(C_{t+1} = \text{Repair}(C_t, P, \tau_t)\)—not a full rewrite—constrained by the plan and evidence map. A sanity check rejects any revision that weakens an evidence-supported validation target (preventing reward hacking). Warm repair (hot-reload in a persistent container) accelerates iteration; the final contract must still pass clean replay from a fresh container to ensure cold-start reproducibility.

Key Contributions

  • Formalization of repository bootstrapping as a reusable startup knowledge problem, with the .bootstrap contract as the agent-consumable artifact.
  • Multi-agent pipeline combining evidence extraction, structured planning, deterministic Docker verification, and delta repair with provenance tracking.
  • Warm repair with clean replay: fast iterative search without sacrificing reproducibility guarantees.
  • Delta repair with sanity check: constrains each repair round to evidence-supported updates, blocking reward hacking via verification weakening.
  • Empirical evaluation on 212 repositories across three benchmarks (ExecutionAgent, Installamatic, Repo2Run) demonstrating downstream cost reduction.

Results

  • Overall bootstrap success rate: 197/212 (92.9%) vs. HerAgent's 179/212 (84.4%)—+8.5 pp (+10.1% relative).
  • ExecutionAgent: 42/50 vs. 41/50 (+2.0%)
  • Installamatic: 38/40 vs. 34/40 (+10.0%)
  • Repo2Run: 117/122 vs. 104/122 (+10.7%)
  • Median generation cost: 43.8K tokens and 24.6 minutes per repository; P90 = 169.2K tokens / 79.0 minutes.
  • Downstream warm-start savings (cold → warm):
  • Median wall time: 346.3s → 249.5s (0.72Ă—); largest gain ExecutionAgent 285.2s → 112.0s (0.39Ă—)
  • Median total tokens: 710.7K → 419.4K (0.59Ă—); largest gain ExecutionAgent 597.9K → 115.7K (0.19Ă—)
  • Ablation: Removing trace-driven repair drops success by 55–70 pp across benchmarks; removing discovery also substantially degrades results; clean replay has smaller aggregate effect but catches residual-state false positives.
  • Warm-state repair time savings: disabling it costs +4,839.9 minutes on Repo2Run alone; per-repository median wall-clock increases of 30–43%.
  • Sanity checks: caught 5/122 reward-hacking cases on Repo2Run, 4/50 on ExecutionAgent, 2/40 on Installamatic.

Limitations

  • Generation cost is non-trivial (median ~44K tokens, ~25 min per repo) and must be amortized over multiple downstream uses; single-use repos gain little.
  • Evaluation uses a fixed base model (DeepSeek-V4-Flash-2026-04-25); generalizability to other LLMs is untested.
  • S (strongest verification) is advisory only—CI-dependent checks (secrets, service containers, specialized hardware) remain non-reproducible locally by design, leaving a gap between local and full CI validation.
  • Clean replay requires Docker; environments with restricted containerization (air-gapped, GPU-only clusters) are not addressed.
  • The framework is evaluated on public benchmark repos; proprietary or highly heterogeneous internal repositories may exhibit different failure modes.

Relevance to Harnesses / Meta-Harnesses

BootstrapAgent is a direct instance of a meta-harness: it wraps lower-level agents (Discoverer, Planner, Generator, Verifier) in a controlled orchestration loop that persists its own outputs as reusable agent-facing artifacts, directly analogous to how meta-harnesses wrap task-specific runners and emit cacheable state. The .bootstrap contract functions as a harness artifact—a structured, verifiable spec consumed by downstream code agents rather than by humans—extending the CLAUDE.md/AGENTS.md pattern into the environment-setup layer. The warm-repair / clean-replay design mirrors the canonical harness principle of fast inner loops with rigorous outer validation. For researchers building harnesses that orchestrate code agents across heterogeneous repositories, this work provides both a concrete architecture and empirical evidence that distilling agent exploration into reusable contracts meaningfully reduces per-task overhead.