Skip to content

Hierarchical Deep Research with Local-Web RAG: Toward Automated System-Level Materials Discovery

🕒 Published (v1): 2025-11-23 05:57 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DToR (Deep Tree of Research) is a locally deployable, hierarchical LLM-agent framework for long-horizon materials and device discovery that combines local-first RAG with adaptive branch-expand/prune tree orchestration. Evaluated across 27 nanomaterials topics against 11 commercial systems, the best local DToR configuration scores 8.57/10 under LLM-as-judge rubrics, outperforming ChatGPT-o4-mini-high (7.96/10) and ChatGPT-o3 (7.95/10) at substantially lower cost.

Problem

System-level nanomaterials discovery (multi-layer stacks, core–shell catalysts, cross-domain sensor platforms) requires long-horizon, hierarchically structured exploration of heterogeneous knowledge sources — a regime where (a) ML surrogates lack coverage, (b) closed-source commercial deep-research agents are costly, opaque, and cannot integrate local/proprietary corpora, and (c) existing single-pass or linear ReAct-style agents lack the breadth–depth coverage needed for combinatorially complex S3–S4-level problems.

Method

Single DR instance: An evidence-first loop that (1) generates a search query, (2) retrieves from a local RAG corpus first (before web, to reduce hallucination/drift), (3) summarizes local evidence, (4) generates a diversity-aware complementary query from the local summary, (5) performs web search for both queries, (6) integrates evidence into a running summary, (7) reflects to propose follow-up queries, repeating until a maximum round limit before (8) finalizing.

DToR controller: Treats each DR instance as a Research Node (RN) in a branch-and-bound tree. A diversifier seeds orthogonal perspectives (P1, P2, …), each spawning a branch with a budget (max depth, nodes per branch, total branches). A router executes the next pending RN, then an analyst decides EXPAND or PRUNE based on budget/quality; if expanding, a knowledge gap explorer materializes new RNs targeting identified gaps at depth+1. Converged or depth-saturated branches are synthesized into perspective reports; a final synthesizer reconciles cross-branch evidence and produces a provenance-rich consolidated report. Conceptually this is tree-of-search over (query, evidence, summary) states, inheriting structured backtracking while targeting citation fidelity under explicit resource budgets.

Local RAG budgets vary (local0/local100/local500 volumes); LLM backbones tested include gpt-oss120B, gpt-oss20B, and QwQ32B (laptop-compatible).

Key Contributions

  • A locally deployable DR framework integrating local-first RAG with open-source LLMs, supporting on-prem deployment without subscription costs.
  • The DToR tree controller providing breadth-then-depth orchestration with adaptive expand/prune and gap-driven branching over single-instance DR.
  • A comprehensive evaluation harness: 16,605 rubric judgments (27 topics × 41 agents × 5 judges × 3 trials), 14,580 pairwise A/B duel decisions, and dry-lab validation via DFT/AIMD simulations on five representative tasks.
  • Full-factorial ablation (Method × LLM backbone × local-corpus budget) isolating orchestration as the dominant performance driver.
  • Identification of "inverse-design hallucination" failure mode — agents proposing chemically incompatible multi-phase systems — pointing to the need for synthesis-aware validator modules.

Results

  • DToR_gpt-oss120B_local500: 8.57/10 overall rubric score (ranked 1st of 41 agents); 79% mean A/B duel win rate.
  • DToR_gpt-oss120B_local100/local0: 8.53/10 and 8.37/10 (both top-3).
  • Single DR, gpt-oss120B: 8.33/10 (4th/5th), still above all commercial baselines.
  • Commercial baselines: ChatGPT-o4-mini-high 7.96, ChatGPT-o3 7.95, ChatGPT-5-thinking 7.85, Gemini 2.5 Pro 7.81, Claude Opus 4 7.78, Grok-3 7.51, Perplexity 7.51.
  • Lightweight local (QwQ32B): 7.80/10, outperforming roughly half of commercial systems.
  • DToR gains over single DR: +0.69 clarity, +0.72 depth, +0.43 novelty, +0.42 applicability on gpt-oss120B_local500.
  • Inter-judge agreement: Pearson r = 0.97, Spearman ρ = 0.97 across 5 LLM critics.
  • Pairwise dueling: DToR agents average 58.6% win rate vs. 52.8% for single DR.
  • Dry-lab (DFT/AIMD): 4 all-metrics-above-baseline events across 5 tasks; gpt-oss local DR leads on 7/10 metrics with aggregate score 98.7.
  • Compute: DToR_gpt-oss120B averages 19.6 h / 4.37 kWh per report on a 4×RTX A5500 workstation; single DR with gpt-oss120B completes in ~30 min.

Limitations

  • Runtime is substantial: full DToR averages 19.6 h per report on high-end workstation hardware (upper bound under serialized scheduling).
  • "Inverse-design hallucinations": agents propose chemically incompatible multi-phase candidates (e.g., hydrophobic fluorosilane + hydrophilic MXene + PEDOT:PSS) because they lack wet-lab feasibility priors, synthesis cost models, or external physics validators.
  • Evaluation relies on LLM-as-judge, which, despite high inter-rater consistency, may share systematic blind spots with the generator models.
  • Dry-lab validation covers only five tasks and uses simulation metrics (DFT/AIMD) rather than experimental synthesis, leaving an empirical gap.
  • Local RAG benefit narrows to near-zero at local0 (no local corpus); a strong backbone alone can close the DToR–single-DR gap in that regime.
  • No demonstrated generalization beyond materials/nanoscience domains.

Relevance to Agentic AI / LLM Agents

DToR is a concrete instantiation of hierarchical multi-agent orchestration — parallel perspective seeding, adaptive expand/prune with explicit budget control, and iterative gap-driven branching — applied to a domain (materials discovery) where single-pass or flat RAG agents fail. The paper directly benchmarks this architecture against production-grade commercial deep-research agents (OpenAI, Gemini, Grok, Perplexity) at scale, providing one of the most rigorous head-to-head evaluations of local vs. cloud agentic systems to date. The identified failure mode of inverse-design hallucination is a generalizable concern for any agentic system operating in constrained physical or logical domains without domain-validator modules. The local-first RAG design choice — prioritizing proprietary/domain-specific corpora before web retrieval — is a reusable pattern for enterprise or privacy-sensitive agentic deployments.