Skip to content

From Sure" toSorry": Detecting Jailbreak in Large Vision Language Model via JailNeurons

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

JailNeurons: fast neuron-based jailbreak detection for LVLMs; ICLR 2026

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

JDJN (Jailbreak Detection via JailNeurons) identifies a sparse set of neurons in LVLMs whose activations causally mediate jailbreak success, then trains a lightweight SVM over multi-layer aggregations of those neuron activations to detect jailbreak inputs. The approach requires only a single forward pass, achieves >99% TPR at <1% FPR on seen attacks, and generalizes to unseen attack types and OOD benign distributions.

Problem

Existing jailbreak defenses for LVLMs are either attack-type-specific (failing to generalize to unseen attacks) or computationally expensive (requiring multiple LLM forward passes or full response generation). Prior mechanistic safety work on "SafeNeurons" explains why aligned models refuse ordinary harmful queries, but does not explain how jailbreak prompts circumvent those mechanisms — a gap that leaves the internal jailbreak signal unexplored and unexploited for detection.

Method

JDJN operates in three stages:

Stage 1 — JailNeuron Localization ("Sure to Sorry"). For each transformer layer \(i\) with activation output \(o_i \in \mathbb{R}^{b \times t \times d}\), a forward hook applies a learnable sparse mask \(m \in [0,1]^d\) via element-wise suppression: \(h(o_i, m) = (1-m) \odot o_i\). The mask is optimized to steer a confirmed jailbreak input toward a refusal token (e.g., "Sorry") while minimizing the number of masked neurons:

\[\delta^* = \arg\min_{\delta \in \mathbb{R}^d} \lambda \| \text{sig}(\delta) \|_1 + \mathcal{L}_\text{CE}(f_i(\text{sig}(\delta), x),\, e_s)\]

Neurons with \(m > \tau\) (e.g., \(\tau = 0.4\)) are designated JailNeurons for that layer.

Stage 2 — Multi-layer aggregation ("Top to Bottom"). Layers are sampled arithmetically at interval \(k\) (selecting \(\lceil l/k \rceil\) layers). JailNeuron activations from these layers are concatenated to form the feature vector.

Stage 3 — Detection. A linear SVM is trained on the concatenated JailNeuron activations from known jailbreak and benign samples. At inference, a single forward pass suffices; upon detection of a harmful prompt, generation is halted immediately.

Key Contributions

  • Empirical demonstration that jailbreak and benign hidden states are linearly separable across layers in LVLMs (extending prior LLM findings to the vision-language setting).
  • A causal mask-based localization algorithm that identifies a sparse subset of neurons (<2% of total when \(\lambda \geq 0.1\)) causally responsible for jailbreak behavior.
  • The JDJN framework: a non-intrusive, data-efficient, single-forward-pass detector that outperforms seven baselines in TPR/FPR and generalizes to OOD attacks and benign distributions.

Results

  • LLaVA, ID (JailBreakV), JDJN1: TPR = 99.7% at FPR ≤ 5%; FigStep: 100%, JAMLLM (OOD): 94.2%.
  • Janus-pro, ID (JailBreakV), JDJN1: TPR = 99.6%; FigStep: 100%, JAMLLM (OOD): 85.3%.
  • JDJN outperforms all seven baselines (JailGuard, ECSO, CIDER, HiddenDetect, GradSafe, JailDAM, AdaShield) on every attack type evaluated.
  • FPR on OOD benign data (MM-Bench, Normal Prompts) remains ≤2% when trained on MM-Vet.
  • Efficiency (LLaVA): JDJN requires 1.02s per sample vs. 84.27s (JailGuard), 15.12s (ECSO), 5.42s (CIDER); faster than vanilla LVLM inference (12.08s) because it short-circuits generation upon detection.
  • JailNeurons constitute <2% of all neurons across models at \(\lambda \geq 0.1\).

Limitations

  • Assumes white-box access to the target LVLM (internal activations and parameter gradients required); not applicable to API-only or black-box deployments.
  • Generalization degrades when training on distribution-limited benign data (e.g., MM-Bench with forced A/B/C/D outputs or text-only Normal prompts); the detector learns spurious correlations rather than semantic harmful content features.
  • Only evaluated on open-source 7B-class models; scalability to larger models or closed proprietary LVLMs is unvalidated.
  • The "sure-to-sorry" localization requires a set of confirmed successful jailbreak samples for training, which may be difficult to obtain in advance for novel attack types.
  • Adaptive attacks specifically designed to evade neuron-level detection are not evaluated.

Relevance to Vision-Language Models

This work directly addresses LVLM-specific jailbreak vulnerabilities, distinguishing the multimodal attack surface (adversarial image perturbations, typography-based attacks, semantic image-text pairing) from text-only LLM threats. The neuron-level mechanistic analysis reveals that jailbreak circumvention of safety alignment leaves identifiable internal signatures distinct from the SafeNeuron circuitry that handles ordinary refusals — a finding with implications for interpretability and red-teaming of VLMs. For researchers tracking VLMs, JDJN establishes that lightweight, architecture-agnostic probes on frozen model internals can match or exceed expensive judge-LLM and preprocessing-based defenses. The generalization results across MiniGPT4, LLaVA, Qwen2-VL, and Janus-pro suggest the JailNeuron phenomenon is a consistent property of safety-aligned LVLMs rather than a model-specific artifact.