Skip to content

ASTRIDE: A Security Threat Modeling Platform for Agentic-AI Applications

🕒 Published (v1): 2025-12-04 13:32 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ASTRIDE is an automated threat modeling platform for agentic AI systems that extends the classical STRIDE framework with a new "A" category for AI Agent-Specific Attacks (prompt injection, context poisoning, unsafe tool invocation). It uses a LLM-orchestrated pipeline where a consortium of fine-tuned VLMs analyze architecture diagrams, and OpenAI-gpt-oss synthesizes their outputs into a final threat model.

Problem

Traditional threat modeling frameworks (STRIDE and variants) rely on manual inspection of system architectures, require significant domain expertise, and lack native support for AI-specific attack surfaces—prompt injection, reasoning subversion, context/memory poisoning, and opaque agent-to-agent communication—leaving critical vulnerabilities undetected in agentic AI deployments.

Method

ASTRIDE has four layers: 1. Data Lake: ~1,200 annotated Mermaid-diagram records (DFDs, component diagrams) labeled with threat vectors, trust boundaries, and mitigations. 2. VLM Consortium: Three VLMs (Llama-3.2-11B-Vision-Instruct, Pix2Struct, Qwen2-VL) fine-tuned via QLoRA using the Unsloth library on ~1,200 diagram records; each independently extracts ASTRIDE threat vectors from an input diagram. 3. Reasoning LLM: OpenAI-gpt-oss ingests a structured composite prompt assembled from all VLM outputs, reconciles conflicts, ranks threats, and produces an explainable final threat model. 4. LLM Agent Layer (OpenAI-Agents-SDK): Orchestrates the full pipeline—generating model-specific prompts for each VLM, collecting outputs, formatting them for gpt-oss, and returning the synthesized report.

Fine-tuning ran on Google Colab (NVIDIA A100/Tesla TPU) in ~27 minutes; peak reserved memory 14.6 GB, actual 5.85 GB; dataset split 2/3 train / 1/6 val / 1/6 test. Deployment uses Ollama for VLM inference.

Key Contributions

  • Extended STRIDE taxonomy with category A (AI Agent-Specific Attacks) covering prompt injection, context poisoning, reasoning subversion, and unsafe tool invocation.
  • First framework to combine fine-tuned VLMs with a reasoning LLM for fully automated, diagram-driven threat modeling of agentic AI systems.
  • Multi-model ensemble strategy for VLM consortium with decision-level synthesis via a reasoning LLM to resolve conflicting predictions.
  • QLoRA-based quantization enabling resource-efficient edge deployment of fine-tuned VLMs via Ollama.
  • Synthetically generated, annotated diagram dataset (Mermaid-format DFDs) for domain-specific VLM training.

Results

  • Qualitative (pre/post fine-tuning on Llama-Vision): Pre-fine-tuning detects 1 vulnerability (prompt injection at prompt processor only); post-fine-tuning additionally identifies context poisoning and unsafe tool invocation with context-aware mitigations.
  • Qualitative (pre/post fine-tuning on Pixtral-Vision): Pre-fine-tuning detects 1 vulnerability (reasoning subversion); post-fine-tuning adds prompt injection, context poisoning, and actionable mitigations.
  • No quantitative precision/recall/F1 numbers or comparison against specific baselines are reported; evaluation is primarily via training/validation loss curves and qualitative before-after prediction comparisons.

Limitations

  • No quantitative benchmark metrics (precision, recall, F1, AUC) reported; evaluation is qualitative and anecdotal.
  • Training/validation loss curves show overfitting signals (validation loss consistently exceeding training loss, loss ratio up to 3.0).
  • Dataset is small (~1,200 records) and synthetically generated, raising questions about generalization to real-world agentic architectures.
  • Evaluation uses only synthetic and the authors' own architecture diagrams; no independent held-out test set from real deployments.
  • No latency/throughput numbers reported for end-to-end inference.
  • Dependency on proprietary OpenAI-gpt-oss as the reasoning layer limits full reproducibility.

Relevance to Harnesses / Meta-Harnesses

ASTRIDE is itself a meta-harness: an LLM agent layer orchestrates a multi-model pipeline (VLM consortium → reasoning LLM), dynamically constructing model-specific prompts, collecting structured outputs, aggregating them into a composite prompt, and synthesizing a final result—exactly the orchestrate-sub-agents-then-synthesize pattern central to meta-harness design. The use of OpenAI-Agents-SDK as the coordination backbone, with each VLM acting as a specialized sub-agent, illustrates how agentic orchestration frameworks can be applied to security-analysis workflows rather than task completion. For researchers tracking harness patterns, ASTRIDE demonstrates a concrete multi-stage pipeline with prompt engineering, ensemble aggregation, and a "judge" LLM—a reusable template applicable beyond threat modeling to any domain requiring structured multi-model consensus.