Skip to content

Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents

🕒 Published (v1): 2026-06-18 20:07 UTC · Source: Arxiv · link

Why this paper was selected

Agent fingerprinting and attribution; critical governance layer for deploying autonomous web agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MARK (Multi-layer Agent fingerprinting framewoRK) is a server-side logging and classification harness that passively fingerprints autonomous web agents across TLS, HTTP/2, and in-browser behavioral layers without any client-side instrumentation. By feeding cross-layer feature vectors into a decision tree, it attributes traffic to specific agent frameworks (AutoGen, Browser Use, Claude, Gemini, Operator, Skyvern), humans, or legacy crawlers with 97% accuracy. The system is deployed on a live instrumented domain and constitutes a reusable measurement infrastructure for agent behavioral analysis.

Problem

Existing defenses against unauthorized AI web agent scraping—robots.txt and active bot-blocking—are widely violated or easily circumvented. No prior work provides multi-class, per-framework attribution of AI web agents from the server side; existing fingerprinting targets binary bot/human decisions and does not account for the structural heterogeneity of LLM-driven agent architectures (vision-based vs. DOM-based vs. accessibility-tree-based).

Method

MARK operates in four stages on a controlled AWS EC2 host running a Go TLS-terminating server:

  1. Trace collection: A live website with five engineered UX scenarios (functional UI discrimination, contextual state retention, hover-triggered interaction, ID/label semantic mismatch, temporal delay adaptability) captures network packets and browser events. The TLS layer logs ClientHello fields and computes JA3/JA4 hashes via fingerproxy/utls. HTTP/2 preludes are recorded in Akamai fingerprint format (SETTINGS frames, WINDOW_UPDATE, pseudo-header order). Client-side events (mousemove throttled at 16 ms, keydown/up, scroll, click) are buffered and flushed via navigator.sendBeacon only at page unload, avoiding trace contamination.

  2. Feature extraction: Four independent extractors produce feature vectors:

  3. Temporal (9 features): Inter-Request Interval (IRI) mean/std/median, request rate, coefficient of variation (CV), dwell time, subresource burst patterns.
  4. TLS (29 features): Post-quantum key exchange fraction, JA4/JA3 decoded fields, cipher/extension counts, HTTP/2 PRIORITY frame stream-3 and stream-5 priority weights, GREASE usage.
  5. HTTP headers (35 features): Sec-Fetch-* rule violations, Sec-Ch-Ua/User-Agent coherence, header ordering hash, Priority header correctness, non-standard custom headers.

  6. Behavioral: Browser event profiles from the per-scenario interaction logs.

  7. Classification: A decision tree classifier trained on the concatenated feature vectors attributes traces to one of nine classes (6 agent types + human + 3 legacy crawlers).

Key Contributions

  • End-to-end programmatic logging harness (MARK) deployable on any instrumented domain; captures TLS, HTTP/2, and behavioral traces without client-side modification.
  • Five controlled UX scenarios specifically designed to surface structural differences in agent perception pipelines (vision, DOM, accessibility-tree).
  • Multi-class agent attribution achieving 97% accuracy; prior work achieved only binary bot/human classification.
  • Empirical characterization showing cross-layer features are complementary: temporal features dominate early (>60% accuracy after 3 requests); behavioral features supplement later; TLS+HTTP isolate Skyvern uniquely via stream-5 priority weight.
  • Dataset of traces from 6 AI frameworks, 3 human baselines, and 3 legacy crawlers under identical task prompts.

Results

  • 97% accuracy for agent-type classification (decision tree, all four feature layers combined).
  • >60% accuracy after only 3 incoming requests using network-level features alone (early-stage identification).
  • Agent-specific findings:
  • AutoGen: User-Agent inconsistency in HTTP headers.
  • Browser Use: Sec-Fetch logic violations.
  • Skyvern: Isolated by TLS/H2 fingerprint alone (distinct TLS extension count, stream-5 priority weight).
  • Claude vs. Gemini: Nearly identical network-layer fingerprints; distinguishable only via event profiles and browsing behavior.
  • Operator: Not separable by TLS/HTTP/behavioral features alone; temporal features enable discrimination.
  • Combining behavioral + browser fingerprinting outperforms either layer alone (consistent with Crichton et al. F1=0.869 baseline on humans).

Limitations

  • Evaluated against only six agent frameworks; architectural diversity of newer or proprietary agents not covered.
  • Agents tested with a single fixed task prompt; behavioral fingerprints may shift under different task types or user-supplied instructions.
  • Decision tree is a simple classifier; adversarial agents that deliberately mimic human TLS/HTTP stacks or randomize request timing could degrade accuracy (evasion not empirically tested).
  • Behavioral feature extraction requires page-end beacon flush; agents that abort sessions early or block sendBeacon reduce behavioral signal.
  • Dataset collected from a single controlled domain operated since April 15th; generalizability to diverse real-world sites not demonstrated.
  • Paper text is truncated; full behavioral extractor feature count and classifier hyperparameters are not visible.

Relevance to Harnesses / Meta-Harnesses

MARK is itself a measurement harness: a deployable, modular logging infrastructure that wraps external agents (treating them as black boxes), instruments their I/O at multiple abstraction layers, and feeds structured traces into a downstream classifier—a pattern directly analogous to meta-harness architectures that wrap agent pipelines for evaluation or monitoring. The staged pipeline (agent configuration → trace logging → feature extraction → attribution) mirrors the probe-execute-evaluate loop common in agent testing harnesses, making MARK a concrete example of server-side agent observability infrastructure. For researchers building harnesses that need to distinguish or audit which underlying agent framework is executing a workflow, MARK's multi-layer feature taxonomy (temporal, TLS, HTTP, behavioral) provides a ready-made signal vocabulary. The finding that cross-layer features are complementary and that network-layer signals suffice for early attribution is directly applicable to harness designs that need lightweight, low-latency agent identification before full behavioral traces accumulate.