Skip to content

SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

🕒 Published (v1): 2026-07-16 17:51 UTC · Source: Arxiv · link

Why this paper was selected

Multi-agent web-search collaboration addressing long-context task tracking; directly buildable system

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SearchOS is a multi-agent framework that reformulates open-domain information seeking as relational schema completion with grounded citations, externalizing search state into four shared data structures to prevent repetitive loops and coverage gaps. It introduces pipeline-parallel agent scheduling, a middleware harness for behavioral enforcement, and a hierarchical skill system. On WideSearch and GISA benchmarks, SearchOS outperforms all single- and multi-agent baselines on recall- and completeness-sensitive metrics.

Problem

Existing tool-integrated LLM agents treat search plans, progress, evidence, and failures as transient conversation content. As interaction histories grow, agents lose track of what has been established, issue redundant queries, and get trapped in repetitive loops when search fails. Multi-agent systems compound this by allowing workers to duplicate effort or leave execution slots idle. There is no persistent, shared mechanism for tracking schema coverage, evidence provenance, or failed search paths.

Method

SearchOS frames the task as relational schema completion with grounded citations: given a natural-language query \(q\), the system constructs a relational search schema \(\mathcal{S} = \{T_m\}_{m=1}^M\) with tables, attributes, primary/foreign keys, and a citation matrix \(C_m\) anchoring each value to a source URL and excerpt.

Four components execute this:

  1. Search-Oriented Context Management (SOCM): externalizes execution state as \(\mathcal{M}_t = (F_t, G_t, C_t, W_t)\) — Frontier Task (dependency-aware task pool), Evidence Graph (atomic findings with provenance tiers and SUPPORT/CONFLICT/REFINE edges), Coverage Map (per-cell fill status with conflict detection), and Failure Memory (typed failure signatures with recurrence counts). Role-specific projections \(\phi_r(\mathcal{M}_t; z_t)\) expose only relevant state to each agent.

  2. Pipeline-parallel orchestration: rather than synchronizing batched dispatches, the system continuously refills freed agent slots with highest-priority unresolved schema gaps: \(D_t = \text{Top}_{\min(b_t, |R_t|)}(R_t; p)\), overlapping explore, search, and writer roles.

  3. Search Tool Middleware Harness: intercepts agent loops at model and tool call boundaries via three components — Context Middleware (composes projected state + retrieved skills before each LLM call), Evidence Extraction Middleware (schema-binds and span-anchors candidates from browser observations before committing), and Sensor Middleware (detects stalls when \(\sum_{j=t-w+1}^{t} \Delta^{\text{cov}}_j = 0 \wedge \sum \Delta^{\text{ev}}_j = 0\) and enforces budget \(\rho_t = \max(n_t^{\text{iter}}/B^{\text{iter}}, n_t^{\text{search}}/B^{\text{search}}, \tau_t/B^{\text{time}})\)).

  4. Hierarchical search skill system: 280 pre-built skills split into orchestrator skills (global playbooks), strategy skills (query reformulation, multi-hop reasoning, stall recovery), and access skills (site-specific retrieval for 138 commercial, 36 government, and 33 organization sources).

Backbone: GLM-5 for agent roles, Qwen3.5-35B-A3B for evidence extraction; Max@3 evaluation with 8 parallel sub-agents and a 1800s wall-clock budget.

Key Contributions

  • Relational schema completion formulation for open-domain information seeking with per-cell citation matrices as a verifiable, measurable objective
  • SOCM: four-component shared persistent state (Frontier Task, Evidence Graph, Coverage Map, Failure Memory) with locked atomic read–modify–write updates
  • Pipeline-parallel continuous dispatch that eliminates batch straggler idle time by immediately reassigning freed slots to coverage gaps
  • Search Tool Middleware Harness that enforces evidence grounding, stall detection, and budget limits outside agent prompts, independent of the backbone model
  • Hierarchical skill system (280 skills) separating reusable strategy skills from site-specific access skills, refined from successful and failed trajectories

Results

  • WideSearch (200 questions, 15+ domains, large-scale table collection):
  • Item F1: 80.3 vs. best baseline A-MapReduce 76.0 (+4.3 pts)
  • Item Recall: 79.7 vs. 74.2 (+5.5 pts); Item Precision: 83.9
  • Row F1: 56.5 vs. Web2BigTable 54.5 (+2.0 pts)
  • GISA (373 queries, four answer formats):
  • Set F1: 76.5 vs. Plan-and-Solve 63.1 (+13.4 pts)
  • Table Item F1: 76.9 vs. ReAct 74.8 (+2.1 pts)
  • Table Row F1: 59.7; List F1: 68.1; Item EM: 50.0 (tied with Web2BigTable)
  • Schema planning ablation (40 multi-table cases): SearchOS autonomous schema planning exceeds oracle-selected fixed schema by +8.2 Item F1 and +7.7 Row F1; fixed multi-table beats fixed single-table on only 21/40 cases

Limitations

  • Evaluation uses Max@3 (best of three runs), which inflates reported performance relative to single-run systems and makes direct comparison with baselines that use single runs ambiguous
  • Skill system coverage is finite (280 pre-built skills); skill synthesis from trajectories is deferred to future work
  • Backbone is proprietary (GLM-5), limiting reproducibility
  • No analysis of latency or cost overhead introduced by middleware and pipeline orchestration relative to baselines
  • GISA Item EM matches but does not exceed the best baseline (50.0), suggesting exact-match full-table reconstruction remains unsolved

Relevance to Agentic AI / LLM Agents

SearchOS directly addresses a core failure mode in long-horizon LLM agents: the collapse of reasoning quality as conversation context grows and implicit state management breaks down. The SOCM framework is a concrete instantiation of the principle that agent coordination state should be explicit, persistent, and shared rather than inferred from prompt history — a design philosophy with broad applicability beyond web search. The middleware harness demonstrates that behavioral safeguards (stall detection, budget enforcement, evidence grounding) are more reliable when implemented at the infrastructure layer than inside model prompts, which is relevant to any tool-use agent architecture. The hierarchical skill system with failure-driven refinement connects to ongoing work on agent self-improvement and reusable tool abstraction in multi-agent systems.