AttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents

🕒 Published (v1): 2025-11-24 10:14 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AttackPilot is a multi-agent LLM framework that autonomously executes four classes of inference attacks (membership inference, model stealing, data reconstruction, attribute inference) against black-box ML services, requiring only minimal user-provided metadata. Powered by GPT-4o, it achieves a 100% task completion rate versus 26.3% for the best general-purpose ML agent baseline, at a mean cost of $0.627 per run while matching near-expert human performance.

Problem

Conducting inference-attack risk assessments of ML services requires non-trivial expertise: selecting semantically appropriate shadow datasets, choosing compatible model architectures, navigating a combinatorial explosion of hyperparameter configurations \(\prod_{i=1}^{n} k_i\), and interpreting probabilistic metrics. This complexity excludes non-expert ML service providers, auditors, and regulators from performing self-assessment. Existing general-purpose agent frameworks (e.g., MLAgentBench) fail at this task, achieving only 26.3% completion due to bad plans, instruction-following failures, task-context loss, and hallucinations.

Method

AttackPilot decomposes the assessment into a two-tier multi-agent architecture:

• ControllerAgent: receives black-box service metadata (endpoint, input/output format, task description), determines which of the four attack types are applicable, and concurrently spawns one AttackAgent per applicable attack. It monitors status and collects final reports.
• AttackAgent: executes the assigned attack end-to-end. Its task-specific action space encapsulates each critical decision as a discrete, guideline-annotated action: Choose Shadow Dataset (semantic label matching, C1), Choose Shadow Model Architecture (similarity–capability trade-off, C2), Set Parameters (dataset size, epochs, batch size, C3/C4), Execute Script, and Final Answer (report generation, C5).

The shared environment provides Linux shells, starter scripts implementing metric-based and neural membership inference, standard model stealing, attribute inference, and inversion-alignment data reconstruction—all requiring only black-box API access. The multi-agent isolation prevents one stuck attack from blocking others and bounds per-agent memory growth. Domain knowledge is embedded as action-level guidelines, directly addressing each of five identified challenge categories.

Key Contributions

• First autonomous agent framework specifically designed for end-to-end inference-attack risk assessment of ML services, with no human intervention after initial service description.
• Task-specific action spaces with embedded domain guidelines for each of the five identified complexity challenges (\(C_1\)–\(C_5\)).
• Multi-agent decomposition (ControllerAgent + concurrent AttackAgents) as an architectural mitigation for bad plans, context loss, and hallucinations documented in prior agent frameworks.
• Trace analysis identifying four error types in MLAgentBench and mapping each to an AttackPilot design component.
• Demonstration of adaptive extension: importance-based query-sample selection under a 3,000-query service constraint, e.g., improving CelebA model-stealing accuracy from 0.348 to 0.420.

Results

• Task completion rate: AttackPilot (GPT-4o) = 100.0% vs. MLAgentBench = 26.3% (5-run average over 20 target services).
• Membership inference accuracy: AttackPilot avg = 0.822, human expert (ML-Doctor) avg = 0.832; gap = −1.0%.
• Attribute inference accuracy: AttackPilot avg = 0.724, expert avg = 0.748; gap = −2.4%.
• Model stealing accuracy: AttackPilot avg = 0.520, expert avg = 0.492; AttackPilot outperforms by +2.8%.
• Data reconstruction MSE: AttackPilot avg = 0.05197, expert avg = 0.05133 (comparable).
• Cost: $0.627/run (≈147,971 input + 25,665 output tokens), 27.11 steps, 17.39 min/run.
• Closed-source vs. open-source: Claude 3.5 Sonnet achieves 100% completion; Mixtral-8×22B and Llama-3.1-70B fail frequently (predominant action loops at 88.6% and 72.9% of trace steps respectively); o3-mini succeeds 20/20, DeepSeek-V3 succeeds 18/20.
• Ablation (Table 5): multi-agent framework alone raises task completion from a lower baseline to 78.0%; full design reaches 100%.

Limitations

• Evaluated on only 20 target services spanning 5 datasets and 4 model architectures; generalization to diverse real-world service types (NLP, tabular, generative models) is unverified.
• Relies on starter scripts that implement a fixed set of attack methods; adding novel attack classes requires manually extending action spaces and scripts.
• Near-expert gap exists for membership inference (−1.0%) and attribute inference (−2.4%), and the human expert baseline uses ML-Doctor defaults without per-case tuning, which likely understates what an expert with optimization effort could achieve.
• Open-source models remain substantially worse; framework effectiveness is tightly coupled to LLM instruction-following capability.
• Adversarial robustness to prompt injection is tested but only superficially discussed in the appendix.

Relevance to Harnesses / Meta-Harnesses

AttackPilot is a concrete domain-specific harness instantiation: it wraps a reusable environment of scripts, datasets, and models behind structured agent action spaces, then coordinates multiple task-specialized agents (AttackAgents) via an orchestrating controller (ControllerAgent)—the canonical meta-harness pattern. The design explicitly documents how the harness architecture mitigates known failure modes of general-purpose agent frameworks (context overflow, bad planning, hallucination), providing empirical ablation evidence for multi-agent decomposition as a harness design principle. For researchers building harnesses in other domains, AttackPilot's approach of encoding domain expertise as discrete, guideline-annotated actions rather than free-form prompts—and its trace-analysis methodology for attributing errors to architectural choices—offers a transferable design and evaluation template.