Skip to content

NVAITC AI Scientist: A Governed End-to-End Research System -- A Hypertension GWAS Case Study

🕒 Published (v1): 2026-07-13 04:53 UTC · Source: Arxiv · link

Why this paper was selected

Governed end-to-end research agent system in biomedical; institutional deployment constraints for harnesses

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

NAIS (NVAITC AI Scientist) is a governed end-to-end agentic research system that orchestrates multi-stage biomedical workflows—from proposal review through pipeline execution to manuscript drafting—while keeping protected health data within institutional boundaries. It validates its architecture on a real-world hypertension GWAS across 286,422 individuals, reproducing known loci after human-AI phenotype reconciliation. The system demonstrates that governed orchestration with scientist-in-the-loop approval gates can match expert-curated analyses without direct PHI exposure.

Problem

Existing agentic research systems target open data, simulated environments, or benchmark tasks; they lack the governance infrastructure required for institutional biomedical settings where protected health information (PHI) must remain within approved boundaries, computational actions must be auditable, and scientific decisions must stay under human oversight. No prior system provided brokered data access, durable multi-day workflow state, and reproducible pipeline orchestration simultaneously.

Method

NAIS composes three cooperating layers:

  1. Proposal-readiness service — a seven-stage NeMo Agent Toolkit pipeline that parses proposals, searches Semantic Scholar/arXiv, evaluates novelty/ethics, and rewrites to an institutional template; each generative stage is wrapped with an evaluator LLM (self-evaluating agent, Algorithm 1) before output is released.

  2. NemoClaw (agentic client) — Nemotron-3 Super (120B, 32,768-token context) running on-premise in an OpenShell sandbox. It maintains durable run state (broker run IDs, logs, repair notes, manuscript sections) persisted across failures and multi-day job queues, rather than chat history. Capabilities are composable skills: SQL cohort extraction, GWAS specification drafting, literature search, document parsing.

  3. Execution layer — two routes: (a) governed/Kubeflow path: NemoClaw submits approved HTTP actions to a broker v2 API that launches isolated PLINK2 GWAS containers; only aggregate artifacts (Manhattan/QQ plots, GLM outputs, QC metrics) return to the agent — no raw SQL rows, no linkable patient IDs, no free-text notes; (b) containerized compute path: direct GPU container discovery for analyses where data can leave institutional boundaries.

Human approval gates are required before SQL cohort materialization and analysis launch. When jobs fail, the agent inspects broker events and pod logs before resubmitting.

Key Contributions

  • NAIS architecture: zero-trust governed agentic system with brokered PHI isolation, durable state, auditability, and aggregate-only artifact return — the first reported system validated on real protected clinical + genotype data.
  • End-to-end GWAS validation: 286,422-individual hypertension GWAS with independent expert comparison, systematic phenotype discordance analysis (3,950 discordant subjects), medication audit, and final replication of established loci.
  • Scientist-in-the-loop characterization: demonstrates that human-AI iterative refinement was necessary for phenotype reconciliation — unconstrained agent autonomy was insufficient for research-grade output.
  • Self-evaluating proposal wrapper: evaluator LLMs wrapping each generative stage caught 852 deficiencies across 18 production proposal-review runs.
  • Secondary DILI case study: drug-induced liver injury prediction via multimodal graph neural network, AUC = 0.842.

Results

  • After team-directed phenotype reconciliation, agent-orchestrated GWAS reproduced all expert-identified genome-wide significant loci: FGF5 (\(-\log_{10} p \approx 70\), strongest signal), ATP2B1, CNNM2, FTO, GRB14 — matching expert cohort in both locus identification and signal direction.
  • Pre-reconciliation discordance: 3,950 subjects misclassified by the initial agent phenotype (BP-threshold vs. ICD-10/Rx definition); 73.7% of these had SBP > 140 or DBP > 90 mmHg, confirming boundary-region systematic error rather than random noise.
  • Post-reconciliation cohort alignment: agent cohort (Control 148,513 / Case 81,076) closely matched expert reference.
  • Proposal-review evaluator: 852 deficiencies detected pre-delivery across 18 production runs.
  • Secondary DILI task: multimodal GNN AUC = 0.842.
  • No novel GWAS discovery claims; results are replications in a Taiwanese cohort.

Limitations

  • No novel scientific discovery demonstrated; validation is replication of known loci, not new hypothesis generation.
  • The agent's scientific reasoning improved primarily through repeated real-world engagement (warehouse schema failures, team corrections), not prompt engineering — suggesting high setup cost per new domain.
  • Phenotype design remains a human-dependent bottleneck; the agent could not autonomously resolve ICD/Rx vs. BP-threshold discordance without directed reconciliation.
  • Durable state and broker infrastructure are tightly coupled to NVAITC/CMUH-specific systems (Kubeflow, broker v2, TPMI imputed library); generalizability to other institutions is undemonstrated.
  • NemoClaw's 32,768-token context limits the scope of multi-step reasoning within a single turn.
  • The self-evaluating wrapper's 852 deficiency count lacks a false-positive/false-negative analysis, so its precision is unknown.
  • Only two task domains validated (GWAS, DILI prediction); broader domain generality is asserted but not shown.

Relevance to Harnesses / Meta-Harnesses

NAIS is a concrete production instance of a governed meta-harness: it wraps domain-specific tools (PLINK2, Kubeflow, SQL extraction) inside a brokered orchestration layer with explicit state durability, human approval gates, and per-stage evaluator LLM wrapping — exactly the structural pattern that defines a meta-harness. The self-evaluating agent wrapper (Algorithm 1) is a reusable harness primitive that enforces quality checks on any generative stage, directly relevant to harness designers seeking to reduce silent failures. The paper's emphasis on durable state over ephemeral chat as an architectural requirement — to survive multi-day job queues and mid-run failures — articulates a key meta-harness design principle that transfers to any long-horizon orchestration system. The human-approval gate pattern, positioned between NemoClaw's plan and broker execution, exemplifies a controlled handoff mechanism that harness engineers must design for when agents operate near sensitive or irreversible actions.