Skip to content

ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; adaptive plug-and-play red-teaming agents against multimodal models

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ARMs is an adaptive red-teaming agent that automatically discovers safety vulnerabilities in VLMs by orchestrating 11 novel multimodal attack strategies via a reasoning-enhanced multi-step planner and a layered memory module with \(\epsilon\)-greedy diversity control. The framework achieves state-of-the-art attack success rates, including >90% ASR on Claude-4-Sonnet, and produces a 30K-instance benchmark (ARMs-Bench) spanning 51 risk categories. Fine-tuning on ARMs-Bench reduces ASR while preserving general model utility.

Problem

Existing VLM red-teaming is limited by: (1) static benchmarks that lag behind evolving architectures and real-world risks, (2) narrow adversarial patterns (often single-modality or manually engineered), and (3) mode collapse in automated methods where attackers over-specialize to a few templates. No prior framework supports controllable, policy-grounded, multimodal red-teaming with scalable diversity.

Method

ARMs operates as an agentic loop: given a risk definition or harmful instruction \(x_i \sim P\), the agent queries a layered memory \(D\) indexed by risk category \(c_r\) and attack strategy \(s_a\), retrieves top-\(k\) past trajectories via cosine similarity \(\text{score}(\zeta) = \cos(\phi(c_r^i), \phi(c_r^\zeta)) + \alpha \cdot \cos(\phi(x_i), \phi(x^\zeta))\), then reasons over retrieved trajectories to select and sequence attack strategies. Each strategy is wrapped as an independent MCP server for plug-and-play integration (17 algorithms total). The agent iteratively refines adversarial multimodal instances \(I_\text{adv}^t = (\text{Image}^t, \text{Text}^t)\) until a policy-based LLM judge scores the victim VLM's response as sufficiently harmful or the budget \(T\) is exhausted. Memory is updated by replacing lower-scoring slots with higher-harmfulness trajectories. Exploration decays via \(\epsilon_i = \epsilon_\text{min} + (\epsilon_\text{max} - \epsilon_\text{min}) \cdot \exp(-\lambda(i-1))\), shifting from broad exploration to exploitation as memory fills. The 11 attack strategies cover five adversarial patterns: visual context cloaking (rule-based, email thread, Slack conversation, news report, scenario playing, narrative), typographic transformation (flowchart, numbered-list image), visual multi-turn escalation (crescendo, actor attack, acronym), visual reasoning hijacking (multimodal trigger backdoor, many-shot mixup, simulated function-call), and visual perturbation (photographic, jigsaw scramble, multimodal shuffling).

Key Contributions

  • First controllable, policy-grounded multimodal red-teaming framework conditioning attack generation on explicit risk definitions.
  • 11 novel multimodal attack strategies organized across 5 adversarial patterns, integrated as MCP servers for modular extensibility.
  • Layered memory with \(\epsilon\)-greedy scheduling to prevent mode collapse and enforce risk–strategy diversity.
  • ARMs-Bench: 30K adversarial instances across 51 risk categories (27,776 single-turn + 2,224 multi-turn), paired with reasoning-enhanced refusals for safety fine-tuning.
  • 1,020-sample static evaluation benchmark (20 instances/category) with policy-based judges for reproducible VLM safety comparison.

Results

  • ARMs improves average ASR by 52.1 percentage points over prior best baseline X-Teaming across six evaluations (three instance-based benchmarks: StrongReject, JailbreakBench, JailbreakV; three policy-based: EU AI Act, OWASP, FINRA).
  • Exceeds 90% ASR on Claude-4-Sonnet across three evaluations, a constitutionally-aligned model.
  • Achieves 95.83% improvement in instance diversity compared to X-Teaming.
  • Fine-tuning on ARMs-Bench reduces ASR with a better robustness–utility trade-off than existing multimodal safety datasets.

Limitations

  • Attack effectiveness requires a query budget \(T\); computationally more expensive than single-pass static attacks.
  • The 17 initial strategies and 51 risk categories, while broad, are still a finite manually curated seed set; genuinely novel zero-day adversarial patterns may not be represented.
  • Policy-based judge quality is dependent on the LLM judge's own alignment and scope; judge errors propagate through the evaluation loop.
  • ARMs-Bench evaluation set (1,020 samples) is static, and ARMs's own dynamic targeting of each VLM makes direct cross-model comparisons potentially inconsistent.
  • White-box perturbation strategies (visual perturbation class) may have reduced transferability compared to semantic strategies.

Relevance to Vision-Language Models

ARMs directly exposes the multimodal attack surface of VLMs that text-only red-teaming misses—cross-modal vulnerabilities like typographic transformation and visual reasoning hijacking are unique to architectures that jointly process image and text. The MCP-integrated plug-and-play design allows the framework to track VLM architectural evolution, addressing the brittle-benchmark problem that afflicts static safety evaluations. ARMs-Bench's reasoning-enhanced refusal pairs offer a practical alignment training resource, connecting safety red-teaming to the broader challenge of robust instruction-following in VLMs. The >90% ASR on a constitutionally-aligned frontier model underscores that current VLM safety alignment remains insufficient against adaptive, strategy-diverse multimodal attacks.