Towards an Agentic Workflow for Internet Measurement Research¶

🕒 Published (v1): 2025-11-13 18:44 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

ArachNet is a four-agent LLM system that automatically generates executable Internet measurement workflows from natural language queries, mirroring the compositional reasoning process domain experts use when integrating heterogeneous measurement tools (BGP analyzers, traceroute processors, topology mappers). It introduces a curated tool registry as a "measurement API" that agents query to compose multi-framework solutions without requiring users to have specialized expertise in each underlying tool.

Problem¶

Internet measurement research requires orchestrating multiple specialized frameworks (each with unique interfaces, data formats, and domain knowledge requirements) into custom workflows. This creates an expert barrier: only a small community with deep specialization can compose sophisticated measurement analyses, and even experts spend days manually integrating tools for complex scenarios like cascading cable failure investigations.

Method¶

ArachNet decomposes expert workflow reasoning into four sequential specialized agents operating over a central Registry: 1. QueryMind — parses a natural language query into structured sub-problems with explicit dependency, constraint, and success-criteria analysis. 2. WorkflowScout — explores the Registry to design solution architectures, applying adaptive depth (direct path for simple queries; multi-approach trade-off analysis for complex ones). 3. SolutionWeaver — converts the architecture into executable Python code, handling data format translation between heterogeneous tools and embedding inline consistency/quality checks. 4. RegistryCurator — analyzes successful workflows, validates reusable patterns, and incrementally extends the Registry with new capability entries.

The Registry encodes tool capabilities in standardized schema entries (capability description, required inputs, expected outputs, constraints) rather than exposing raw codebases, scaling linearly with the tool count. An optional expert mode allows human review between any two adjacent agent stages.

Key Contributions¶

First demonstration of LLM agents independently generating Internet measurement workflows that match expert-level reasoning without domain-specific architectural guidance.
A curated, standardized Registry abstraction that exposes heterogeneous measurement frameworks as a unified "measurement API" for agent composition.
A four-stage multi-agent architecture that separates problem decomposition, solution exploration, implementation, and capability evolution into distinct reasoning phases.
Validation across three progressive complexity levels: expert-solution replication, multi-framework orchestration (4 frameworks, ~525 LoC), and temporal forensic root-cause investigation (~750 LoC).
Open-sourced prompts and case studies.

Results¶

Level 1 (Expert Replication): ArachNet independently replicates Xaminer's country-level cable impact analysis (~250 LoC) without access to Xaminer's higher-level abstractions; achieves functionally equivalent logic for multi-disaster analysis (~300 LoC) by correctly avoiding over-engineering.
Level 2 (Multi-Framework Orchestration): Automates cascading Europe–Asia cable failure analysis across 4 frameworks (~525 LoC), replacing what traditionally requires days of manual multi-system coordination.
Level 3 (Forensic Analysis): Implements temporal correlation + causation establishment for submarine cable root-cause identification (~750 LoC), integrating statistical anomaly detection, infrastructure correlation, and BGP validation.
No quantitative accuracy metrics (e.g., precision/recall, latency) are reported; validation is qualitative functional equivalence to expert solutions.

Limitations¶

Evaluation is confined to Internet resilience case studies within a single domain; generalization to other measurement domains (application performance, security monitoring) is unvalidated.
No quantitative correctness metrics; functional equivalence is assessed qualitatively against expert reference implementations.
Generated code sometimes contains minor programming errors (not domain-knowledge errors), requiring post-hoc debugging.
The Registry is manually curated, introducing a bootstrapping bottleneck for new domains; RegistryCurator only extends it incrementally after successful runs.
Trust and verification for novel queries without expert ground truth remain open problems.
Dependence on specific LLM capabilities (Claude Sonnet) raises portability questions across other model architectures.

Relevance to Harnesses / Meta-Harnesses¶

ArachNet is a direct instance of a domain-specific meta-harness: it wraps a collection of existing tools (BGP analyzers, traceroute processors, topology mappers) behind a Registry abstraction and uses a multi-agent pipeline to generate, validate, and evolve tool-composition workflows automatically. The RegistryCurator agent introduces a self-extending harness pattern — the system grows its own tool registry from successful runs, which is a concrete implementation of the "harness that builds harnesses" concept. The separation of concerns across QueryMind → WorkflowScout → SolutionWeaver mirrors the plan/design/implement decomposition common in general agentic harness architectures, offering a reusable template for domain-specific harness construction. The Registry-as-API design (capability schemas rather than raw code) is directly applicable to any harness that must integrate third-party tools with heterogeneous interfaces.