Skip to content

ToFu: A White-Box, Token-Efficient Agent Harness for Researchers

🕒 Published (v1): 2026-07-13 11:26 UTC · Source: Arxiv · link

Why this paper was selected

White-box agent harness paper; directly buildable; token-efficiency and orchestration transparency

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ToFu is an open-source, MIT-licensed agentic coding harness designed as both a practical research assistant and a white-box research instrument. It achieves higher SWE-bench Verified pass rates than Claude Code while consuming 28.4% fewer tokens on average. Its three-layer context compaction, multilingual translate-then-reason pipeline, and dependency-aware swarm scheduler are its core technical contributions.

Problem

Current agentic harnesses fall into two dysfunctional camps: research-oriented harnesses are reproducible but benchmark-shaped and lack a usable product surface, while commercial products (Claude Code, Codex) default to proprietary, server-side-updated models that are poor research instruments. Both camps suffer from excessive token consumption, poor multilingual performance (since LLM English-centrism propagates directly to user-facing quality), and barriers to local/private deployment.

Method

ToFu is a six-module harness (User Interface, Agent Orchestration Core, Model Abstraction Layer, Capability Runtime, State & Knowledge, Context Management) built around three core techniques:

  1. Three-layer context compaction: (i) Size-aware budgeting externalizes large tool outputs (search results, command output) as compact previews with recoverable references; (ii) Cache-aware micro-compaction replaces older bulky tool outputs and stale reasoning blocks with concise placeholders deterministically, without an extra LLM call, and without modifying cached prefixes; (iii) Query-aware semantic compaction triggers when the context limit approaches—a lightweight model (e.g., GPT-4o) summarizes older turns conditioned on the current query, preserving critical turns at high fidelity and discarding irrelevant ones.

  2. Translate-then-reason multilingual mode: Non-English user input is translated to English before the main reasoning step, then the output is translated back. Code blocks and marked non-translatable spans are protected during translation.

  3. Dependency-aware swarm scheduler: Sub-tasks form a directed acyclic graph (DAG); independent agents run in parallel while dependent agents wait for prerequisites. A master agent in reactive mode periodically decides whether to spawn additional agents based on accumulated evidence.

Memory uses BM25 ranking over structured Markdown records (with metadata: name, description, tags, scope) rather than dense embeddings, keeping retrieval latency and cost low.

Key Contributions

  • Open-source (MIT) agentic harness that functions both as a usable assistant and as a white-box baseline for harness research.
  • Three-layer context compaction that reduces average token usage by 28.4% vs. Claude Code (up to 43.6% on individual LLMs) while achieving higher Pass@1.
  • Translate-then-reason multilingual mode yielding +2.5 accuracy points on average across 10 languages on MAPS:SWE-bench vs. direct source-language prompting.
  • DAG-structured dependency-aware swarm scheduler for parallel multi-agent coordination.
  • MCP integrations enabling direct interaction with Overleaf, GitHub, Slack, and Notion; Claude Code itself is callable as a sub-tool.

Results

  • SWE-bench Verified (500 instances), Claude Opus 4.6 backbone: ToFu 83.2% vs. Claude Code 79.6% vs. OpenCode 74.2%; ToFu cost $1.19/inst. vs. $5.13/inst. for Claude Code.
  • SWE-bench Verified, GLM 5.1 backbone: ToFu 80.4% vs. Claude Code 77.6% vs. OpenCode 71.6%.
  • SWE-bench Verified, DeepSeek-V4-Pro backbone: ToFu 80.2% vs. Claude Code 75.2% vs. OpenCode 71.8%.
  • Averaged over all three LLMs: ToFu outperforms Claude Code by +3.8 pp and OpenCode by +8.7 pp in Pass@1.
  • Average token reduction vs. Claude Code: 28.4% (up to 43.6%).
  • MAPS:SWE-bench multilingual (10 languages, Claude Opus 4.6): multilingual enhancement yields +2.5 pp average accuracy; positive in 7 of 10 languages.

Limitations

  • Evaluation is dominated by agentic coding (SWE-bench); broader research-assistant evaluation is limited to a small human preference study with only three participants.
  • No systematic quantitative analysis of the token-efficiency vs. task-performance trade-off.
  • Multilingual enhancement shows slight declines in 3 of 10 languages.
  • The semantic compaction layer introduces a dependency on an auxiliary LLM (e.g., GPT-4o), adding latency and cost when triggered.

Relevance to Agentic AI / LLM Agents

ToFu directly addresses a structural gap in the harness research ecosystem: the absence of an open-source baseline that simultaneously achieves state-of-the-art benchmark performance and a production-grade user experience, making it a suitable substrate for harness evolution research (meta-harness, self-evolving agents, observability-driven optimization). Its empirical finding that more tokens do not yield better performance challenges the implicit assumption behind test-time scaling in agentic settings, with implications for how researchers should evaluate efficiency-performance trade-offs. The DAG-based swarm scheduler and BM25-based memory retrieval offer concrete, inspectable design points for multi-agent coordination and long-term memory, both active frontiers in the field. The multilingual mode is directly relevant to expanding agentic AI access beyond English-dominant research contexts.