Skip to content

ARMOR++: Agentic Orchestration of a Multi-Domain Primitive Set for Transferable Attacks on Deepfake Detectors

🕒 Published (v1): 2026-07-16 17:45 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ARMOR++ is a multi-agent, multimodal adversarial framework that uses a VLM (Qwen2.5-VL) for semantic spatial priors and an LLM (Qwen3) for closed-loop orchestration to generate highly transferable adversarial examples that evade deepfake detectors under a strict no-query black-box setting. It extends the prior ARMOR system with two new perturbation primitives (SSA, BSR) targeting frequency-domain and patch-based inductive biases of ViT/Swin detectors. Evaluation on AADD-2025 shows significant Attack Success Rate (ASR) gains over both agentic and non-agentic baselines.

Problem

Existing transfer-based adversarial attacks on deepfake detectors (MI-FGSM, DI-FGSM, AutoAttack) fail to generalize across architectural boundaries—specifically from CNN surrogates to transformer-based targets—because they lack semantic awareness and rely on a purely spatial perturbation space. Prior agentic work (ARMOR) used only three spatial primitives, permitted a black-box probability term \(p_{bb}\) in mixing that violates the no-query constraint, and was evaluated only on one low-quality regime against a single blind target.

Method

ARMOR++ operates as a four-phase closed loop per outer iteration \(k\):

  1. Semantic Reconnaissance (Phase 1): Qwen2.5-VL-32B generates a structured report \(\mathcal{R}^{(0)}\) identifying facial topology, texture irregularities, and forensic-vulnerability regions. This is converted to a soft spatial prior \(\widetilde{M}_R \in [0,1]^{H \times W}\) (Eq. 8). A Qwen3-32B Conductor Agent initializes global state \(S^{(0)} = (\epsilon^{(0)}, \tau^{(0)}, \{\boldsymbol{\phi}_m^{(0)}\}, \mathbf{w}^{(0)})\).

  2. Parallel Primitive Generation (Phase 2): Five Method Agents execute concurrently over the primitive set \(\mathcal{M} = \{\text{CW, JSMA, STA, SSA, BSR}\}\), each producing a candidate \(\delta_m^{(k)}\):

  3. CW-style: \(\ell_2\)-penalized dense optimization with semantic-masked gradient \(\widetilde{M}_R \odot \nabla_\delta J_\text{CW}\)
  4. JSMA: sparse saliency attack with VLM-reweighted pair score \(S_\text{JSMA}(p,q) = \widetilde{M}_p \widetilde{M}_q s^+_{pq} - s^-_{pq}\)
  5. STA: geometric deformation via differentiable bilinear warp with total-variation flow regularizer \(L_\text{flow}\)
  6. SSA: frequency-domain augmentation via DCT with random spectral modulation \(A_\omega = 1 + \sigma\omega\), \(\sigma\omega \sim \mathcal{N}(\mathbf{0}, \rho^2 I)\), optimizing Monte Carlo expectation over \(Q_\text{SSA}\) samples

  7. BSR: block-structured augmentation with random block permutation \(\pi \in S_{N_b}^\text{nbhd}\) and per-block rotations, targeting patch-based ViT inductive biases

  8. Critique and Reparameterization (Phase 3): A deterministic Critique Agent scores candidates via \(Q^{(k)}(\mathbf{z}) = \mu_p \bar{p}_\text{surr}(t|\mathbf{z}) - \mu_\ell \tilde{L}_\text{surr}(\mathbf{z},t) - \mu_s[\tau^{(k)} - \text{SSIM}(\mathbf{x},\mathbf{z})]_+\). A Strategist Agent detects stagnation and relaxes \((\epsilon, \tau)\) bounds; an Advisor Agent proposes hyperparameter updates \(\Delta\boldsymbol{\phi}_m^{(k)}\) via LLM reflection.

  9. Entropy-Regularized Mixing (Phase 4): A Mixer Agent selects convex combination weights \(\mathbf{w}^{*(k)} \in \Delta^{|\mathcal{M}|}\) to synthesize \(\mathbf{x}^{(k)}_\text{master} = P_{\epsilon^{(k)}}(\mathbf{x}, \sum_m w_m^{(k)} \delta_m^{(k)})\) without querying the blind target \(g\).

The surrogate ensemble uses \(R=3\) CNNs (ResNet-50, DenseNet-121, EfficientNet-B4) with weighted ensemble loss \(L_\text{surr} = -\sum_r \alpha_r \log p_{f_r}(t|\mathbf{x})\). The blind target is queried exactly once, post-hoc, for reporting only.

Key Contributions

  • Strict no-query transfer protocol: All optimization, selection, and mixing rely solely on surrogate feedback; the mixing score contains no \(p_{bb}\) term, unlike ARMOR.
  • Five-primitive multi-domain perturbation space: Adds SSA (frequency) and BSR (block-structured) to the three ARMOR primitives (CW, JSMA, STA), targeting spectral and patch-based inductive biases of ViT/Swin detectors.
  • VLM-guided semantic priors: Qwen2.5-VL supplies a soft spatial mask \(\widetilde{M}_R\) that reweights gradients and saliency scores toward forensic-vulnerability regions.
  • Entropy-regularized five-way mixer: Convex combination of all primitive candidates without target feedback.
  • Reliability-oriented statistical evaluation: ASR values reported with 95% Wilson confidence intervals; pairwise improvements assessed by exact McNemar tests with Holm–Bonferroni correction.
  • Cross-regime evaluation: LQ and HQ image regimes on AADD-2025, two transformer blind targets (ViT-B/16, Swin-B), and a zero-shot check on DFDC-Preview.

Results

  • ARMOR++ "significantly outperforms existing agentic and non-agentic baselines across both low- and high-quality image regimes" on AADD-2025 (specific ASR numbers were in the truncated portion of the paper).
  • "Statistical analysis confirms a substantial gain in blind-target ASR over the state-of-the-art agentic baseline [ARMOR]."
  • "Further performance advantages evidenced against non-agentic benchmarks [AutoAttack-PGD] and under robust defensive configurations."
  • Component-removal ablations isolate contributions of: semantic analysis, reparameterization, stagnation handling, entropy-regularized mixing, SSA, BSR, and the third surrogate (EfficientNet-B4).
  • Two blind targets evaluated: ViT-B/16 and Swin-B; zero-shot transfer also tested on DFDC-Preview.

Limitations

  • The provided text is truncated before the full results tables; specific ASR values, confidence intervals, and ablation numbers are not available for verification here.
  • Evaluation is restricted to face-domain deepfake detectors on AADD-2025 and DFDC-Preview; generalization to non-facial AI-generated image detectors is untested.
  • The VLM (Qwen2.5-VL-32B) and LLM (Qwen3-32B) introduce substantial compute overhead relative to static attacks; per-image surrogate forward-pass counts are reported but not reproduced in the excerpt.
  • The SSIM constraint is enforced as a soft penalty in the mixing score, not as a hard projection, so SSIM threshold \(\tau\) may be violated on individual candidates.
  • The surrogate ensemble is limited to CNNs, so gradient diversity across architectures is partial; ViT surrogates are not included.

Relevance to Vision-Language Models

ARMOR++ is a direct demonstration of VLMs as active components in adversarial attack pipelines: Qwen2.5-VL supplies structured forensic-region descriptions that are converted into differentiable spatial priors shaping gradient computation across all five attack primitives. This positions VLMs not as attack targets but as semantic perception modules that close the gap between architecture-agnostic pixel-level optimization and forensically meaningful perturbation placement. For researchers tracking VLMs, the paper documents a concrete integration pattern—VLM analysis feeding soft spatial masks into gradient-based optimization—that is distinct from prompt-injection or jailbreaking attack vectors. The framework also illustrates how VLM-derived spatial understanding can substitute for target-model queries in black-box settings, a transferable design principle for other VLM-in-the-loop agent architectures.