Skip to content

NOVA: An Agentic Framework for Automated Histopathology Analysis and Discovery

🕒 Published (v1): 2025-11-14 14:01 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

NOVA is a coding-agent framework that translates natural language queries into iteratively executed Python pipelines for computational histopathology analysis, integrating 49 domain-specific tools (nuclei segmentation, WSI encoding, MIL training, etc.). The authors also introduce SlideQuest, a 90-question pathologist-verified benchmark demanding multi-step reasoning, tool orchestration, and hypothesis testing on whole-slide images. NOVA (GPT-4.1 backbone) achieves an overall score of 0.477 on SlideQuest, substantially outperforming all coding-agent baselines.

Problem

Digitized histopathology workflows require multi-step processing pipelines and narrow specialized tools, creating a gap between the questions researchers want to ask and their ability to execute them without deep programming expertise. Existing medical AI benchmarks only evaluate text-based knowledge recall or static single-image VQA, failing to capture iterative coding, tool orchestration, and dataset-scale analysis. No prior benchmark supports evaluation of computational agents performing multi-step quantitative histopathology workflows.

Method

NOVA is built on CodeAct (Wang et al., 2024) and implemented via smolagents. A core LLM receives a dynamically constructed system prompt combining general code-generation instructions, tool docstrings, and user-provided special instructions. It produces structured JSON with thought and code fields; code is executed in a Python 3.11 interpreter and results are fed back into the LLM context. This loop runs for up to 20 iterations per query, acting as a per-task scratchpad (memory is cleared between queries except during conversational case studies). The 49 custom tools—organized into seven categories covering ROI analysis, nuclei segmentation/classification, tissue detection, patch extraction, feature computation, and full supervised MIL experiments—are implemented as atomic Python functions with standardized docstrings, built on open-source computational pathology packages. NOVA can also synthesize additional tools ad hoc using standard data science libraries when none of the 49 tools suffice.

Key Contributions

  • NOVA: a modular CodeAct-based agentic framework that dynamically writes and executes Python to build histopathology analysis pipelines from natural language, requiring no instruction-fine-tuned models.
  • A library of 49 atomic, open-source-backed histopathology tools covering data ingestion through supervised WSI-level MIL experiments.
  • SlideQuest: a 90-question publicly released benchmark verified independently by a pathologist and a biomedical scientist, spanning DataQA, CellularQA, PatchQA, and SlideQA across 33 labelled capability dimensions.
  • Quantitative ablations showing custom tools are essential (e.g., +0.240 on DataQA vs. no-tools variant; RAG over documentation achieves only 0.337 overall vs. 0.477 with custom tools).
  • Failure case taxonomy (tool limitations, framework limits, ignored tools/data, LLM fabrications) and a pathologist-verified PAM50 case study.

Results

  • NOVA (GPT-4.1) overall SlideQuest score: 0.477; baselines: LLM only = 0.000, LLM+PI = 0.154, LLM+PI+retries = 0.269, establishing a +0.208 margin over the best baseline.
  • Per-category: DataQA = 0.777, CellularQA = 0.323, PatchQA (implied intermediate), SlideQA = 0.472 (GPT-4.1) / 0.551 (GPT-5).
  • Custom tools vs. no custom tools: +0.240 DataQA, +0.171 CellularQA, +0.113 PatchQA, +0.033 SlideQA.
  • RAG-based tool creation: overall 0.337 vs. 0.477 with hand-crafted tools.
  • GPT-5 improves harder categories (SlideQA: 0.551 vs. 0.472) but degrades easier ones (DataQA: 0.708 vs. 0.777) and substantially increases runtime (47.4 h vs. 31.2 h on SlideQA).
  • Full 90-question sequential run: 40 hours on a single NVIDIA A100.

Limitations

  • SlideQuest evaluation checks only final outputs; incorrect intermediate reasoning, fabricated data, or random guessing are not penalized.
  • The framework cannot distinguish between errors from tool implementations vs. agent misuse of tools.
  • 49 tools cannot anticipate all edge cases and may themselves contain bugs, limiting scalability.
  • High run-to-run variance (agentic stochasticity) makes guaranteed reproducibility difficult.
  • TCGA data carries known demographic biases; SlideQuest coverage is currently limited to breast cancer (BRCA).
  • Operation-limit safeguard in the Python interpreter causes premature termination on compute-heavy tasks, forcing the agent to fall back to data subsets.

Relevance to Harnesses / Meta-Harnesses

NOVA is a concrete domain-specific harness instantiation: it wraps an LLM backbone inside a code-execution loop with a curated tool registry, a structured system-prompt builder, and iteration control—exactly the architectural pattern of a meta-harness that dynamically composes sub-tools into task-specific pipelines. The ablations directly quantify the value of carefully engineered tool libraries vs. RAG-generated or LLM-improvised tools, providing empirical evidence that harness design (tool curation, atomic decomposition, docstring standardization) dominates backbone model choice on domain-specific benchmarks. SlideQuest's multi-step, tool-orchestration evaluation design is itself a harness-evaluation methodology transferable to other domains: it benchmarks not the model in isolation but the full harness behavior under iterative execution. The failure taxonomy (exceeding interpreter limits, ignoring available tools, fabrications) maps directly to known failure modes in general meta-harness design.