Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare

🕒 Published (v1): 2026-06-27 01:00 UTC · Source: Arxiv · link

Why this paper was selected

Security evaluation framework for healthcare agents; TRiSM-guided methodology

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Applies the Gartner AI Trust, Risk, and Security Management (TRiSM) framework to transform an insecure single-agent medical report generation workflow into a security-conscious multi-agent workflow, then benchmarks both against injection attacks across five LLMs. The TRiSM-guided design reduces mean attack success by 21–32 percentage points across three vectors and improves report accuracy by 14 pp, while showing that model choice alone is insufficient without architectural security controls.

Problem

Agentic AI systems are routinely granted access rights that exceed those of ordinary users, yet integrated without systematic security governance — often exacerbated by "vibe coding" practices that produce functional but insecure agent logic. Existing mitigations address individual vulnerabilities (injection, inference, memory poisoning, tool misuse) in isolation but rarely provide a combined architectural response. For medical applications, this risks patient data exposure, regulatory non-compliance, and clinically dangerous outputs, and the literature lacks empirical demonstrations of a unified framework applied to a working medical agentic system.

Method

Compares two architectures on the same Mendix-based medical report generation application: (1) an insecure agent workflow — a single agent with access to all tools and resources, constructing prompts client-side via non-persistable objects; and (2) a TRiSM-guided agentic workflow — multi-agent decomposition (Patient Search, Document Generation, Similar X-ray agents) with data partitioning, server-side prompt construction, per-agent tool scoping, embedding analysis, and policy JSONs.

Evaluation across five LLMs (Claude Haiku 4.5, GPT-4.1-nano, GPT-4.1-mini, GPT-5.4-mini, Gemini 2.5 Flash) on two report types (standard + Covid-19 encounters), totalling 800 generations. Security testing covers three attack vectors (RAG poisoning, data-field injection, client-side network injection) with 20 attack scenarios per vector per workflow/model combination. The agentic workflow renders the network injection vector inapplicable by constructing prompts server-side, yielding 500 total attack executions. Attack success is defined as any output deviation toward the injection's intent. Results reported with 95% Wilson score intervals and McNemar's exact test.

Key Contributions

• First empirical application of TRiSM to an operational medical agentic application, mapping each of the five principles (Governance, Trustworthiness, Fairness, Reliability, Data Protection) to concrete architectural controls.
• Demonstrates that the TRiSM-guided redesign reduces attack success across three vectors and five models while simultaneously improving report accuracy.
• Identifies vulnerability classes that emerge when LLM agents are granted access beyond the underlying user's permissions, with per-model attack-success benchmarks.

Results

• RAG poisoning: mean attack success reduced from 31% to 10% (21 pp reduction; \(p<0.001\), McNemar).
• Data-field injection: mean attack success reduced from 42% to 25% (17 pp reduction; \(p<0.05\)).
• Client-side network injection: eliminated entirely (100% → 0%, inapplicable due to server-side prompt construction).
• Report accuracy: increased from 72.5% to 86.5% (14 pp improvement) with the agentic workflow.
• The underlying insecure non-persistable-object data pattern means stored prompts are accessible via network traffic interception, nullifying database-level access controls.
• Per-model cells (\(n=20\) per vector) are reported but individually underpowered for fine-grained ranking.

Limitations

• Single medical application in a single low-code platform (Mendix), limiting generalisability.
• The TRiSM-guided condition is the full redesign bundle; individual component impact (e.g., embedding analysis vs. data partitioning) is not isolated.
• Attack set is application-specific rather than a general benchmark, trading breadth for clinical relevance.
• Per-model per-vector sample sizes (\(n=20\)) are too small for reliable model-level comparisons.
• Paper text appears truncated before Section 5 (recommended workflow details) and any discussion of limitations, so additional caveats may be unobserved.

Relevance to Harnesses / Meta-Harnesses

This paper treats TRiSM as a meta-harness — a governance framework that orchestrates the security posture of agentic workflows rather than a single tool or benchmark. It shows concretely that wrapping a multi-agent pipeline in a security framework with least-privilege data partitioning, server-side prompt control, and per-agent tool scoping yields measurable gains against injection attacks. For the meta-harnesses topic, it provides an empirical template for evaluating how a governing framework modifies the security properties of an agentic system, though it stops short of isolating which components of the TRiSM harness drive the improvement.