Skip to content

DiscoVerse: Multi-Agent Pharmaceutical Co-Scientist for Traceable Drug Discovery and Reverse Translation

🕒 Published (v1): 2025-11-23 03:17 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DiscoVerse is a production multi-agent system deployed at Roche that orchestrates specialized LLM agents—Preclinical, Clinical, Strategic, Supervisor, Review, Research, and Taxonomy—over 0.87B-token confidential pharmaceutical archives spanning 180 drug molecules and four decades. It is designed for reverse translation: extracting decision-critical insights from discontinued-program documents to inform current drug development. Across seven benchmark queries it achieves near-perfect recall (\(\geq 0.99\)) with precision of \(0.71\)–\(0.91\).

Problem

Pharmaceutical companies accumulate heterogeneous, multi-decade document repositories (study reports, clinical trial data, toxicology findings, strategic decisions) from discontinued programs. Synonym drift, CRO-introduced formatting variation, scanned legacy PDFs, and siloed sources make keyword search infeasible and single-agent LLMs unreliable. The result: reverse translation—reusing archived knowledge to inform new programs—is practically impossible at scale despite immense institutional value.

Method

DiscoVerse implements a hierarchical multi-agent pipeline with three parallel domain branches (Preclinical, Clinical, Strategic), each internally composed of: 1. Decomposition Agent: rewrites the user query into sub-queries using a read–rewrite–retrieve (\(R^3\)) pattern. 2. Search Agent: hybrid retrieval (symbolic + dense semantic) against a BPE-indexed corpus. 3. Review Agent: reranker-score filtering (threshold 0.7) + LLM relevance judgment; rejects non-relevant chunks and loops back to decomposition if gaps remain. 4. Research Agent: prompt-based evidence extraction and synthesis from filtered chunks.

A Supervisor Agent orchestrates the three branches and merges findings. A Classification Agent routes queries to the appropriate branches and maps them to question-type categories co-designed with scientists. A Taxonomy Agent downstream maps synthesized findings into a schema library—a curated set of structured output templates co-authored by project leads—enabling consistent, auditable answers. A rule-based classifier fallback handles LLM routing failures.

Key Contributions

  • Role-specialized agent architecture that mirrors scientist workflows: Preclinical, Clinical, and Strategic branches with dedicated Decomposition, Search, Review, and Research sub-agents.
  • Human-in-the-loop integration: expert-in-the-loop schema library co-designed with toxicologists and clinical scientists for structured, auditable output.
  • Expert evaluation protocol on real, confidential pharmaceutical R&D data (180 molecules, \(>0.87\)B tokens, \(>40\) years) with per-query TP/TN/FP/FN rubrics—not synthetic benchmarks.
  • First agentic framework systematically assessed for reverse translation on authenticated pharmaceutical archives.

Results

  • Across seven benchmark queries (Q1–Q7) over 180 drug molecules:
  • Recall: \(\geq 0.99\) (near-perfect)
  • Precision: \(0.71\)–\(0.91\) (query-dependent)
  • Qualitative assessments on three real-world pharmaceutical use cases showed faithful, source-linked synthesis across preclinical and clinical evidence.
  • Q8 (Discontinuation Rationale) and Q9 (Multi-Phase Toxicity Integration) were not scored quantitatively across all programs; evaluated via case studies only.
  • Baselines: no comparison to alternative RAG or single-agent LLM baselines is reported; evaluation is absolute (expert adjudication against ground-truth source documents).

Limitations

  • No comparison against single-agent RAG or GPT-4 baselines; absolute recall/precision figures cannot be contextualized relative to simpler systems.
  • Q8 and Q9—the most strategically complex queries—were not quantitatively evaluated across all 180 molecules.
  • Precision of \(0.71\)–\(0.91\) implies meaningful false-positive rates in a safety-critical domain; sources of FPs not characterized.
  • Evaluation relies entirely on expert human adjudication, which is non-reproducible and resource-intensive; scalability of this evaluation protocol is limited.
  • Data is proprietary and confidential; no public benchmark or dataset is released, preventing reproducibility or community comparison.
  • Schema library requires ongoing co-design with domain experts; coverage of novel or unanticipated query types is unaddressed.

Relevance to Harnesses / Meta-Harnesses

DiscoVerse is a concrete instance of a domain-specific meta-harness: a supervisor-orchestrated pipeline that composes specialized sub-agents—each with its own retrieval, review, and synthesis logic—under a shared routing, schema, and fallback layer. The pattern of Classification → Decomposition → domain branch → Review loop → Supervisor merge → Taxonomy structured output is directly analogous to harness architectures that coordinate heterogeneous agents under a central orchestrator. Notably, the schema library and rule-based fallback in the Classification Agent exemplify harness-level concerns: deterministic guardrails layered over probabilistic LLM components to ensure pipeline reliability. The human-in-the-loop schema co-design models how domain knowledge can be encoded into harness configuration rather than baked into model weights, a generalizable pattern for harness designers working in regulated or knowledge-intensive domains.