Skip to content

Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction

🕒 Published (v1): 2026-04-29 21:43 UTC · Source: Arxiv · link

Why this paper was selected

Bi-level multi-agent system for internet-scale structured information extraction

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Web2BigTable is a bi-level multi-agent framework for web-to-table search that separates orchestration (task decomposition) from execution (parallel worker agents), with both levels evolving via a closed-loop run–verify–reflect process over persistent external skill banks. It achieves 7.5× improvement over the previous best on the WideSearch benchmark while also generalizing to depth-oriented search (73.0% on XBench-DeepSearch). Skill adaptation occurs entirely through human-readable external memory without fine-tuning the underlying LLMs.

Problem

Existing agentic web search systems fail at either breadth (wide-coverage structured extraction across many entities) or depth (multi-hop reasoning over a single target). Monolithic agents saturate their context windows at scale, and hierarchical frameworks with fixed planning lack feedback loops that co-adapt decomposition strategies and execution skills together. Prior self-evolving agents only adapt at a single level, not jointly across orchestrator and worker tiers.

Method

Web2BigTable instantiates a bi-level policy \(\tau = (\tau_1, \ldots, \tau_N) \sim \pi_o(\cdot | q, S_o)\) and \(x_i \sim \pi_w^{(i)}(\cdot | \tau_i, m_e, s_i),\ s_i \in S_w\), separating an upper-level orchestrator from a pool of lower-level parallel workers.

Memory architecture: - Long-term semantic memory: two persistent skill banks \(S_o\) (orchestrator decomposition strategies) and \(S_w\) (worker execution skills), stored as human-readable SKILL.md files; monotonically updated during training, frozen at inference. - Short-term working memory: a per-query shared workboard \(m_e\) (Markdown file with file locks and tag-partitioned regions) through which concurrent workers exchange partial findings asynchronously.

Training (self-evolving): A run–verify–reflect loop over \(K\) training episodes refines both banks in parallel. After each episode, trajectories are compressed into a structured error report \(r_o^{k+1}\), which feeds two reflect operators: \(S_o^{k+1} = M_o(S_o^k, r_o^{k+1})\) and \(S_w^{k+1} = M_w(S_w^k, r_o^{k+1})\). Orchestrator skills are clustered by structural decomposition pattern (e.g., split-by-entity, split-by-time-period) and a task-router skill is learned. Worker skill resolution uses a three-tier prioritized search: exact name match → hybrid BM25+ChromaDB embedding retrieval (BAAI/bge-m3 + RRF + cross-encoder) over 8,000+ pre-configured skills → on-demand LLM synthesis (Python function or Markdown knowledge skill), with AST validation and error-driven self-repair.

Inference: Frozen banks \((S_o^*, S_w^*)\) are consumed read-only; no reflection occurs. The orchestrator classifies the query via a task-router and partitions it into \(N\) subtasks (each scoped to 10–20 items to stay within worker context limits); workers execute in parallel using \(S_w^*\) and the shared workboard.

Key Contributions

  • Bi-level architecture separating orchestrator (decomposition) and worker (execution) tiers, interacting only through shared external memory.
  • Joint bi-level self-evolution: the run–verify–reflect loop simultaneously refines \(S_o\) (decomposition strategies) and \(S_w\) (execution skills) without LLM fine-tuning.
  • Asynchronous shared workboard enabling parallel workers to avoid redundant exploration, reconcile conflicting evidence, and fill coverage gaps mid-episode.
  • Three-tier SkillResolver (exact match → semantic retrieval → on-demand synthesis) with error-driven self-repair and live index refresh propagated across all workers.
  • Evaluation showing generalization across structurally distinct benchmarks (breadth: WideSearch; depth: XBench-DeepSearch).

Results

  • WideSearch (wide/breadth):
  • Avg@4 Success Rate: 38.50 vs. 5.10 (second best) — 7.5× improvement
  • Row F1: 63.53 (+25.03 over second best)
  • Item F1: 80.12 (+14.42 over second best)
  • XBench-DeepSearch (depth): accuracy 73.0%, demonstrating generalization to single-target deep reasoning without architectural changes.

Limitations

  • Training requires gold-standard tables, limiting applicability to tasks where ground-truth structured outputs are available.
  • The paper text is truncated and does not report ablations over worker pool size \(N\) or maximum steps \(T_{\max}\), leaving sensitivity analysis unclear.
  • Skill banks grow monotonically (append-only); no pruning or conflict resolution mechanism is described for skills that may become stale or contradictory over long training runs.
  • The orchestrator uses GPT-5 mini as the reasoning LLM; generalization across other backbone models is not characterized.
  • Coordination via shared Markdown files with file locks may introduce latency bottlenecks as the worker pool scales.

Relevance to Harnesses / Meta-Harnesses

Web2BigTable is a concrete instance of a bi-level meta-harness: the orchestrator tier functions as a planning harness that decomposes tasks and selects strategies, while the worker tier is a parallel execution harness with shared coordination primitives. The run–verify–reflect loop is itself a meta-harness pattern — a training harness wrapping the inference harness to iteratively improve both levels without touching model weights. The use of persistent, human-readable SKILL.md files as the adaptation substrate makes skill banks version-controllable and externally editable, a property directly relevant to anyone building composable harness libraries. The clean separation between a frozen inference harness and a training meta-harness that evolves its configuration is an architectural template applicable beyond web search.