Skip to content

TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization

🕒 Published (v1): 2026-03-26 05:56 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TopoPilot is a two-agent agentic framework that automates multi-step topological data analysis and visualization workflows via natural language, achieving >99% success on 1,000 simulated multi-turn conversations. It separates workflow construction (orchestrator agent) from pre-execution validation (verifier agent), using a custom node-tree representation and deterministic guardrails to enforce structural and semantic correctness without runtime code generation.

Problem

Existing agentic visualization systems (VizGenie, ChatVis, ParaView-MCP) rely on code generation or unconstrained MCP tool use, making them brittle: syntactically valid but semantically wrong outputs, unhandled ambiguous prompts, lack of pre-execution validation, and poor extensibility. Topological data analysis workflows are especially complex (multi-stage preprocessing → descriptor computation → tracking), yet no prior agentic system reliably targets them. No prior system had undergone rigorous reliability evaluation across failure modes or adversarial/infeasible inputs.

Method

TopoPilot uses a two-agent architecture built on the Model Context Protocol (MCP):

  1. Orchestrator agent: Receives the user's natural language prompt, asks clarifying questions for underspecified inputs (F1 mitigation), then invokes atomic MCP tools to build a node tree — a custom DAG-like data structure encoding the computation pipeline. Each node corresponds to one deterministic backend operation (e.g., PersistenceSimplification, ComputePersistenceDiagram). Deterministic guardrails constrain permissible node types, parameter ranges, and operation sequences inline.

  2. Verifier agent: Receives the completed node tree and deterministically validates it for (a) structural validity (legal operation sequences), (b) parameter correctness (F3), (c) capability scope (F2), (d) workflow coherence (F4), and (e) alignment with the user's stated goal (F5). Validation is deterministic — not LLM-based — so it enforces hard correctness guarantees before any computation runs.

Upon passing verification, the workflow is deterministically executed by the Python backend (ParaView + Topology Toolkit). Intermediate results are cached to enable parameter adjustments without upstream recomputation. Finalized workflows export as portable Python scripts using predefined macros, satisfying reproducibility requirements. The six failure modes (F1–F6: clarification failure, capability confusion, invalid parameterization, invalid workflow, goal misalignment, erroneous execution) each have an explicit safeguard mapped to them.

Key Contributions

  • Reliability-focused two-agent architecture with separation of interpretation (orchestrator) and verification (verifier), eliminating runtime code generation overhead and errors.
  • Node tree representation: a custom hierarchical data structure for multi-step visualization pipelines with built-in deterministic validity constraints.
  • Failure mode taxonomy: systematic enumeration of six failure classes (F1–F6) with targeted guardrails for each, covering the full lifecycle from prompt parsing to execution.
  • Extensible MCP backend: new topological descriptors or visualization libraries are added as Python subclasses of an abstract base class without modifying core system logic.
  • Large-scale reliability evaluation: 1,000 simulated multi-turn conversations across 100 prompts (including adversarial and infeasible requests), raising success rate from <50% to >99%.

Results

  • Overall success rate: >99% with full guardrails vs. <50% baseline (no guardrails), across 1,000 simulated multi-turn conversations on 100 prompts.
  • Evaluation covers adversarial and infeasible requests in addition to standard prompts — the first such rigorous multi-failure-mode evaluation for an agentic visualization system.
  • Prior systems (VizGenie, ChatVis) reported failure rates on 80 and 40 trials respectively, limited to fully specified, single-turn prompts only — not comparable in rigor.

Limitations

  • Input data restricted to VTK image format (.vti); other scientific data formats are not supported.
  • 2D vector and tensor field support only; 3D vector/tensor fields are absent.
  • Transfer-function refinement for volume rendering is intentionally lightweight (selector only, no iterative NL-driven tuning).
  • Evaluation is simulation-based (LLM-simulated conversations), not a user study with real domain scientists.
  • The verifier's deterministic checks must be manually authored per node type; coverage gaps could allow new failure modes as the backend grows.
  • No quantitative ablation of individual guardrails (F1–F6 contributions are not individually isolated in the reported numbers).

Relevance to Harnesses / Meta-Harnesses

TopoPilot is a domain-specific execution harness with a formal failure-mode taxonomy and deterministic validation layer sitting between the LLM and tool execution — precisely the architecture pattern central to reliable meta-harness design. The separation of orchestration from verification mirrors the harness/validator split seen in general agentic harness literature, and the node tree is a concrete implementation of a typed, inspectable intermediate representation that prevents invalid action sequences before they run. The six-failure-mode taxonomy and the >99% vs. <50% ablation provide empirical grounding for the value of pre-execution validation guardrails, a principle transferable to any harness coordinating multi-step LLM tool use.