Skip to content

DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Sociotechnical audit framework for citation reliability in deep research LLM agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DeepTRACE is an audit framework for evaluating generative search engines (GSEs) and deep research (DR) LLM agents across eight dimensions covering answer text, sources, and citations. Findings across GPT-4.5/5, Perplexity, You.com, Copilot/Bing, and Gemini show that DR configurations reduce overconfidence but remain highly one-sided on debate queries and exhibit unsupported statement rates as high as 97.5%, with citation accuracy ranging 40–80%.

Problem

Existing benchmarks for RAG-based systems evaluate isolated components (retrieval quality, summarization faithfulness, claim-level factuality) rather than end-to-end agent behavior. There is no systematic framework that jointly audits how DR agents retrieve sources, structure URL-level citations, express confidence, and balance perspectives—gaps that directly affect millions of end users who trust these systems for information synthesis.

Method

DeepTRACE operationalizes 16 community-identified failure modes (from a prior usability study) into 8 automated metrics. For each system response, the framework performs five preprocessing steps: 1. Statement decomposition — answers are split into atomic statements, each labeled for query relevance and pro/con stance (for debate queries). 2. Confidence scoring — an LLM judge assigns a 1–5 Likert confidence score (Pearson \(r=0.72\) vs. human annotators). 3. Source scraping — full text of cited URLs is extracted via Jina.ai Reader; ~15% of URLs are inaccessible and excluded. 4. Citation Matrix — a binary (statements × sources) matrix recording which statements cite which URLs. 5. Factual Support Matrix — an LLM judge fills a parallel (statements × sources) binary matrix indicating whether each source actually supports each statement (Pearson \(r=0.62\) vs. human labels).

From these matrices, eight metrics are derived: - One-Sided Answer (binary): answer lacks both pro and con statements on a debate query. - Overconfident Answer (binary): one-sided and confidence score = 5. - Relevant Statement ratio: \(\frac{\text{relevant statements}}{\text{total statements}}\). - Uncited Sources ratio: fraction of listed sources appearing in at least one citation. - Unsupported Statements ratio: fraction of relevant statements with no factual support from any listed source. - Source Necessity ratio: minimum vertex cover (Hopcroft-Karp algorithm on the bipartite factual-support graph) divided by total listed sources. - Citation Accuracy: \(\frac{\sum (\text{Citation Mtx} \odot \text{Factual Support Mtx})}{\sum \text{Citation Mtx}}\) - Citation Thoroughness: \(\frac{\sum (\text{Citation Mtx} \odot \text{Factual Support Mtx})}{\sum \text{Factual Support Mtx}}\)

The corpus contains 303 questions: 168 debate queries (from ProCon.org) and 135 expert-contributed research questions.

Key Contributions

  • A sociotechnically grounded audit framework (DeepTRACE) with 8 formally defined, automated metrics covering answer confidence/balance, source relevance, citation accuracy, and factual support.
  • Statement-level decomposition + dual matrix representation (Citation Matrix, Factual Support Matrix) enabling fine-grained traceability of evidence use.
  • A 303-question corpus (debate + expert domains) used to benchmark GPT-4.5/5, Perplexity, You.com, Copilot/Bing, and Gemini in both GSE and DR configurations.
  • Empirical finding that listing more sources does not improve grounding: high Uncited Sources and high Unsupported Statement rates coexist in leading commercial systems.
  • Release of evaluation dataset and automated extraction pipelines (browser scripts for four GSE platforms).

Results

  • One-sidedness on debate queries (DR mode): GPT-5(DR) 54.7%; YouChat(DR) 63.1%; Copilot(DR) 94.8% — majority of answers still one-sided despite deep-research configuration.
  • Unsupported statement rate (DR mode): YouChat(DR) 74.6%; Perplexity(DR) 97.5% — nearly all statements in Perplexity's DR output lack support from its own cited sources.
  • Citation accuracy: 40–80% across all evaluated systems.
  • DR vs. GSE: DR configurations reduce overconfidence and increase citation thoroughness for some models, but do not resolve one-sidedness or unsupported-statement problems.
  • LLM-judge reliability: confidence scoring Pearson \(r=0.72\); factual support Pearson \(r=0.62\) vs. human annotators.
  • Roughly 80,000 factual support checks were performed in the evaluation.

Limitations

  • Factual Support Matrix relies on LLM judging with only moderate human agreement (\(r=0.62\)); results are treated descriptively rather than as ground truth.
  • ~15% of URLs are inaccessible (paywalled or 404), excluded from support calculations, potentially biasing results.
  • Browser-scraping pipelines are system-specific; extending coverage to additional GSEs requires per-site engineering.
  • Metrics do not cover answer completeness, coherence, or synthesis quality — only sourcing and traceability.
  • UI-level failure modes (identified in the prior usability study) are excluded because they require computer vision or manual evaluation.
  • Corpus is limited to 303 questions; generalization to other domains or query types is not demonstrated.

Relevance to Agentic AI / LLM Agents

DeepTRACE directly targets a core trust problem in agentic research systems: the gap between the appearance of grounding (citations, source lists) and actual factual support. As LLM agents are increasingly deployed for autonomous web research with multi-step retrieval and synthesis, this framework provides the first end-to-end audit methodology that tracks evidence through the entire pipeline—retrieval, citation, confidence expression—rather than at isolated stages. The finding that even best-in-class DR agents (GPT-5 Deep Research, Gemini Deep Research) exhibit substantial one-sidedness and unsupported statements is a direct empirical constraint for anyone building or deploying research agents. The bipartite Source Necessity formulation using Hopcroft-Karp is a technically novel addition that could be reused in agent evaluation harnesses for measuring source redundancy and retrieval efficiency.