Skip to content

TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging

🕒 Published (v1): 2026-06-29 04:17 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TopoAgent is an LLM-based agentic framework that automates the selection and configuration of topological descriptors for medical image classification, replacing manual expert-driven trial-and-error. It operates via a Perception–Reasoning–Action–Reflection (PRAR) loop backed by 21 domain-specific tools, a distilled skill set, and dual short/long-term memory. On TopoBenchmark (113,182 samples, 26 datasets), it achieves 68.21% average balanced accuracy, outperforming the best fixed-descriptor baseline by 9.32% and general-purpose LLMs with the same tools by over 21%.

Problem

No automated method exists for selecting which topological descriptor (e.g., persistence images, persistence landscapes, Minkowski functional) best suits a given medical image dataset. The choice depends on subtle interactions between dataset morphology and each descriptor's mathematical properties; no single descriptor dominates across datasets (Table 1a confirms this across five representative datasets). Prior LLM-based medical agents cannot compute persistent homology (PH) or translate topological observations into validated descriptor configurations, and general-purpose LLMs with tool access still produce suboptimal choices due to lack of empirical grounding. No standardized cross-descriptor benchmark existed.

Method

TopoAgent wraps Claude (or any LLM backbone) in a four-phase PRAR loop:

  1. Perception — six deterministic tools compute the PH profile \(h(I)\) (birth-death counts, average persistence, Betti ratio \(\beta_1/\beta_0\), per-dimension distributions), visual statistics \(v(I)\) (SNR, contrast, edge density), and object type \(o \in \{\text{cells, glands/lumens, organ shapes, vessel trees, surface lesions}\}\) via joint image+PH reasoning.

  2. Reasoning — two-step with deliberate asymmetric information access to combat anchoring bias:

  3. Proposal step: LLM sees \(h(I)\), \(v(I)\), \(o\), and descriptor properties \(S_\text{prop}\) (definitions, strengths, weaknesses) plus stripped reasoning patterns from \(S_\text{rank}\) (without actual rankings) → proposes \((d_p, \theta_p)\).
  4. Determination step: LLM integrates full \(S_\text{rank}\) (tiered rankings, reasoning chains, PH signal thresholds) and long-term memory \(M_l\) → retains, defers to rankings, or applies a prior correction.

Withholding full \(S_\text{rank}\) during proposal reduces the top-ranked agreement rate from 92.1% (anchored) to 61.3%, of which 38.8% are independently correct and 22.5% are valid corrections by \(S_\text{rank}\).

  1. Action — deterministic invocation of the chosen descriptor tool from the 15-descriptor library with parameters drawn from validated \(S_\text{param}\) (not LLM-generated), producing \(f \in \mathbb{R}^{n_d}\).

  2. Reflection — LLM assesses \(f\) via summary statistics (sparsity, variance, kurtosis, skewness, dynamic range, informative-feature ratio) against descriptor-specific reference ranges from \(S\). On failure, a structured correction is written to \(M_l\) and the pipeline retries from the determination step (max 2 retries), falling back to the top-ranked descriptor for \(o\).

The skill set \(S\) is constructed offline: (A) grid search over 75 descriptor–object-type combinations for \(S_\text{param}\); (B) 2,340 balanced accuracy evaluations (\(26 \times 15 \times 6\) classifiers) to derive \(S_\text{rank}\); (C) five domain experts formulate qualitative \(S_\text{prop}\) grounded in TDA principles.

Dual memory: \(M_s\) = short-term (within-run context window history); \(M_l\) = long-term structured diagnostic entries (failed descriptor, diagnosed cause, successful correction), accumulated within a dataset and reset between datasets.

Key Contributions

  • TopoAgent: first agentic framework integrating LLM reasoning with TDA, enabling automated per-image topological descriptor selection without task-specific training.
  • TopoBenchmark: first standardized benchmark for cross-descriptor evaluation — 26 datasets, 113,182 samples, 5 object types, 11 imaging modalities, convergence-based per-dataset sample sizing.
  • Distilled skill set \(S\): offline-derived from systematic evaluation (2,340 BA values), encoding descriptor properties, tiered per-object-type rankings with reasoning chains, and validated parameter configurations.
  • Two-step reasoning with asymmetric information to prevent anchoring bias in LLM descriptor selection.
  • Reflection with diagnostic memory: self-correction loop that writes structured entries to \(M_l\) and enables within-dataset adaptation.

Results

  • Overall: 68.21% average balanced accuracy across 26 datasets.
  • vs. best fixed descriptor baseline: +9.32% (the strongest single-descriptor baseline).
  • vs. general-purpose LLMs with same tools (MedRAX/GPT-4o, Claude, Gemini): +21% average balanced accuracy.
  • Object-type identification: joint PH+vision identification substantially outperforms PH-only or vision-only (Table 2, exact numbers not quoted in excerpt).
  • Anchoring ablation: full \(S_\text{rank}\) at proposal time → 92.1% top-ranked agreement (degenerate lookup); stripped proposal → 61.3% top-ranked agreement, with 38.8% independent agreement and 22.5% valid rank-based corrections.

Limitations

  • Evaluation is restricted to 2D medical images and PH computed via sublevel-set filtration on pixel intensities (cubical complexes); extension to 3D volumes or other filtration types is not addressed.
  • The skill set \(S\) is organized by object type (5 categories), which may not generalize to morphologies outside this taxonomy or to new imaging modalities not represented in TopoBenchmark.
  • Long-term memory \(M_l\) resets between datasets, so cross-dataset transfer of diagnostic experience is explicitly disabled; the agent cannot learn from failures on related datasets.
  • Reflection validates feature vector quality (sparsity, variance, etc.), not downstream task accuracy — the agent has no signal about classification performance at inference time.
  • Maximum of 2 retries before fallback; pathological inputs that exhaust retries revert to the top-ranked descriptor, potentially losing the adaptive benefit entirely.
  • Skill set construction required significant expert effort (5 domain experts, 2,340 evaluations, grid search); updating \(S\) for new descriptors or modalities is costly.

Relevance to Harnesses / Meta-Harnesses

TopoAgent is a domain-specific meta-harness: it wraps a library of 15 specialist tools (topological descriptor implementations) inside a structured PRAR loop that reasons about which tool to invoke, configures it correctly, and validates its output — exactly the pattern of a harness that orchestrates subordinate tools rather than calling them directly. The distilled skill set \(S\) plays the role of a harness-level knowledge base that encodes when each sub-tool is appropriate, analogous to routing logic in a meta-harness. The dual-memory design (short-term for within-run state, long-term for cross-run learning) and the Reflection-driven retry loop are reusable harness primitives that could be adopted in other domain-specific orchestration frameworks. For researchers tracking harness design, the anchoring-bias mitigation via asymmetric information exposure is a notable contribution — it shows that the information schedule presented to the LLM within a harness phase materially affects whether the orchestrator makes independent decisions or degenerates into a lookup table.