Skip to content

SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

🕒 Published (v1): 2026-07-16 17:51 UTC · Source: Arxiv · link

Why this paper was selected

Multi-agent collaboration for long-horizon search; addresses context/progress tracking in orchestration

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SearchOS is a multi-agent framework that reformulates open-domain information seeking as relational schema completion with grounded citations, externalizing all search state into a shared structure called SOCM (Search-Oriented Context Management). A middleware harness intercepts model and tool boundaries to enforce budgets, ground evidence, and detect stalls without modifying agent prompts. On WideSearch and GISA benchmarks, it outperforms all single- and multi-agent baselines.

Problem

Long-horizon search agents treat plans, progress, evidence, and failure records as transient context — losing track of what is resolved vs. missing, duplicating effort across parallel workers, and entering repetitive loops when search paths fail. Prompt-level safeguards cannot reliably cover these failure modes across heterogeneous post-trained agents, so stall detection and budget enforcement must be moved outside agent prompts.

Method

The system has four interlocking components:

  1. Relational schema formulation. A natural-language query \(q\) generates a schema \(\mathcal{S} = \{T_m\}_{m=1}^M\) of normalized tables with primary/foreign keys. The task is to fill a value matrix \(Y_m \in \mathcal{V}^{N_m \times |A_m|}\) and a citation matrix \(C_m\) anchoring each cell to a source URL and span. Coverage is measured as \(\text{Cov}(\mathcal{C}_t) = \frac{\sum_{c \in \Omega_t} \mathbf{1}[s(c)=\text{FILLED}]}{|\Omega_t| + \sum_{m: |E_m(t)|=0} |A_m|}\).

  2. SOCM. Four shared, locked memories: Frontier Task (dependency-aware task pool \(F_t\)), Evidence Graph \(G_t\) (atomic provenance nodes with SUPPORT/CONFLICT/REFINE edges), Coverage Map \(C_t\) (per-cell status with lexicographic evidence selection over provenance tier, schema alignment, and confidence), and Failure Memory \(W_t\) (typed failure signatures with recurrence counts). Agents receive role-specific projections \(x_t^{(r)} = \phi_r(M_t; z_t)\) rather than full state.

  3. Pipeline-parallel orchestration. A continuous event-driven dispatcher fills \(b_t\) execution slots with top-priority ready tasks \(D_t = \text{Top}_{\min(b_t, |R_t|)}(R_t; p)\), releasing and immediately refilling slots on completion to avoid batch straggler idle time.

  4. Search Tool Middleware Harness. Three-stage pipeline wrapping each agent step: (a) Context Middleware — projects SOCM state and retrieves top-\(K\) skills before each LLM call; (b) Evidence Extraction Middleware — binds and anchors extraction candidates from observations, accepts \(z\) only if \(\text{Bind}(z,S)=1 \wedge \text{Anchor}(z, o_t)=1\), then atomically commits to Evidence Graph and Coverage Map; (c) Sensor Middleware — measures \(\Delta^\text{cov}_t\) and \(\Delta^\text{ev}_t\) over a window \(w\), declares a stall when both are zero for \(w\) steps, and computes budget pressure \(\rho_t = \max(n_t^\text{iter}/B^\text{iter},\, n_t^\text{search}/B^\text{search},\, \tau_t/B^\text{time})\) to trigger correction, backfill, drain-only, or stop actions.

  5. Hierarchical search skills. 280 pre-built skills organized into orchestrator (global playbooks), strategy (source-independent query methods), and access (site-specific retrieval with typed executors) tiers. A query-driven router selects and injects relevant skills per task.

Key Contributions

  • Relational schema completion with grounded citation matrices as a unified, measurable objective for open-domain search.
  • SOCM: four-component shared external state (Frontier Task, Evidence Graph, Coverage Map, Failure Memory) with locked atomic updates and role-scoped projections.
  • Continuous pipeline-parallel orchestration replacing synchronized batch dispatch.
  • Search Tool Middleware Harness enforcing evidence grounding, stall detection, and budget limits outside agent prompts.
  • Hierarchical 280-skill system (strategy + access tiers) persisted across sessions.

Results

  • WideSearch (200 questions, table-structured answers): Item-F1 80.3, Row-F1 56.5, +4.3 Item-F1 over strongest baseline (A-MapReduce, 76.0).
  • GISA (373 queries, four answer formats): Set-F1 76.5 (+13.4 over best baseline at 63.1), Table Item-F1 76.9, Table Row-F1 59.7, List-F1 68.1, Item-EM 50.0.
  • Gain concentrated in recall-sensitive metrics, consistent with coverage-aware dispatch design.
  • Ablation on 40 multi-table-eligible cases: SearchOS (Item-F1 70.6) beats oracle fixed-schema selection (62.4) by +8.2 points.

Limitations

  • Reports Max@3 (best of three runs), which inflates comparison to single-run baselines.
  • Backbone models (GLM-5, Qwen3.5-35B-A3B) are not open-weight; reproducibility depends on API access.
  • Skill synthesis at scale (from trajectories and user intent) is deferred to follow-up work; the 280 pre-built skills were manually curated.
  • Entity discovery in open-set tables measures coverage over known rows, not true recall against an exhaustive ground truth.
  • No ablation isolating the Middleware Harness contribution from SOCM or skill system.

Relevance to Harnesses / Meta-Harnesses

SearchOS is a direct instantiation of the harness pattern: the Search Tool Middleware Harness wraps agent execution at LLM call and tool-call boundaries without altering agent logic, enforcing evidence grounding, stall detection, and budget limits as system-level invariants — exactly the "meta-level control plane" design that distinguishes harnesses from in-prompt orchestration. Unlike harnesses that merely route or schedule, this one actively modifies context (pre-call), extracts and commits structured evidence (post-tool), and triggers behavioral interventions (sensor), making it a rare example of a three-stage middleware harness with formal state semantics. The SOCM shared state infrastructure also illustrates how a harness can maintain durable cross-agent memory that individual agent contexts cannot, a key open problem in multi-agent meta-harness design.