Skip to content

Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysis

🕒 Published (v1): 2026-07-15 08:49 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Analogical Deep Research (ADR) is a new task requiring LLM agents to retrieve structurally similar historical events and integrate them for foresight analysis. Existing deep research (DR) agents fail because they match on surface features rather than causal mechanisms. The proposed CANA agent, grounded in two formal identifiability theorems, achieves >10% gains over state-of-the-art self-reflection baselines.

Problem

Current DR agents (OpenAI Deep Research, Gemini Deep Research, Qwen Deep Research, MiroFlow) do not proactively exploit historical analogies in foresight tasks, and when they do find analogies, the analysis remains shallow (level-2 surface claims). Theorem 4 (Surface Non-Identifiability) proves that surface-level retrieval cannot distinguish mechanism-distinct worlds: \(\inf_{\hat{Z}_s} \max_j \Pr_{w_j}(\hat{Z}_s \neq Z_s(w_j)) \geq 1/2\), so any purely descriptive forecast incurs a non-vanishing foresight risk.

Method

CANA (Causal Analogical Researcher) implements two theory-derived principles:

  1. Structural decomposed representation (Principle 1): LLMs decompose each event into preconditions, temporal chain, mechanism (\(M(E) = (V_E, \mathcal{E}_E)\) as a causal DAG), and outcome—enabling retrieval over \(M(E)\) rather than the surface description \(D(E)\).

  2. Structural reflective generation (Principle 2): Iterative self-critique prompts the LLM to cover uncovered structural positions until each hidden position \(s\) is confirmed by \(K_s^* \geq 2\) independent analogies, exploiting the Bayesian cross-analogy confirmation result (Theorem 5): each independent confirming analogy multiplies posterior odds by \(q_{k,s}/p_{k,s}\), so false-positive probability decays exponentially.

CANA synthesizes a Structural Analogy Brief (SAB)—containing mechanism statements, per-analogy limitations, cross-analogy invariants, and predicted hidden factors—and injects it as additional context into the MiroFlow DR multi-agent framework as a lightweight plug-in module.

Key Contributions

  • First formal definition of the ADR task and a causal-theoretic framework with two identifiability theorems justifying design choices.
  • ADR-bench: 15 events (5 financial, 6 geopolitical, 4 technology) with temporal cutoffs, oracle analogies, cross-analogy invariants, and hidden factors; spans 10 historical and 5 forward-looking (unresolved) events.
  • Five-level claim taxonomy (L1 surface fact → L4 hidden factor inferred from ≥2 cross-analogy sources) enabling fine-grained evaluation of analogy depth.
  • CANA agent integrating structural decomposition + reflective generation into MiroFlow, demonstrating modular composition with an existing DR harness.

Results

  • CANA MDS score (GPT-5.4-mini): 4.01 vs. self-reflection 3.34, summarizing 3.59, direct 3.05.
  • CANA MDS score (Sonnet-4.6): 4.17 vs. self-reflection 4.11, summarizing 3.91.
  • CANA MDS score (Qwen3-8B): 4.11 vs. self-reflection 3.94, summarizing 3.47.
  • On 9-dimension structured rubric (Final score, GPT-5.4-mini): CANA 5.22 vs. self-reflection 4.12; (Sonnet-4.6) CANA 5.70 vs. self-reflection 5.23; (Qwen3-8B) CANA 4.87 vs. self-reflection 4.63.
  • Preliminary ADR-bench evaluation shows most DR agents produce zero structural (L3-S/L4) claims; only Claude produces any structural claims at all.
  • (ADR-bench aggregate results on CANA vs. baselines are in a truncated portion of the paper.)

Limitations

  • ADR-bench is small (15 events), limiting statistical power; forward events lack ground-truth outcomes for quantitative verification.
  • CANA relies on LLM parametric knowledge for analogy retrieval—no external knowledge base is searched for historical events, potentially missing obscure analogies.
  • Structural decomposition quality is LLM-dependent; no ablation on the decomposition step itself is reported.
  • The Mechanism Transfer assumption (Assumption 3) is borrowed from historical practice but not empirically validated across domains within the paper.
  • Evaluation is rubric-based with GPT-5.4 as judge, introducing potential self-serving bias and limiting reproducibility.

Relevance to Harnesses / Meta-Harnesses

CANA is explicitly designed as a plug-in module for an existing DR multi-agent harness (MiroFlow), illustrating the pattern of composing specialized sub-agents into a larger orchestration framework—precisely the design space that harness/meta-harness research addresses. The SAB artifact functions as a structured inter-agent communication protocol, passing mechanistic context from a retrieval sub-agent to a synthesis agent, demonstrating how harnesses can be extended via auxiliary context-generation stages without modifying the core agent loop. The paper also benchmarks four production DR harnesses (OpenAI, Gemini, Qwen, MiroFlow) against a novel task class, providing concrete evidence that current harness architectures lack domain-specific retrieval scaffolding and that modular augmentation outperforms end-to-end retraining.