Skip to content

Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations

🕒 Published (v1): 2025-11-24 09:38 UTC · Source: Arxiv · Venue: NEURIPS 2024 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents a taxonomy of LLM jailbreak defenses and proposes three complementary strategies: a prompt-level sanitization/paraphrasing pipeline, a training-free logit-based activation steering mechanism, and a MetaGPT multi-agent pipeline enforcing role-based domain adherence. The agent-based defense achieves zero attack success rate on the evaluation set, while the other two methods provide meaningful but partial reductions.

Problem

LLMs remain vulnerable to adversarial jailbreak prompts that bypass safety filters. Existing defenses are scattered across disparate intervention points (prompt, model internals, training) with no unified comparative framework, and single-layer defenses are insufficient against adaptive attacks. Domain-specific LLM agents face the additional risk of topic deviation under prompt injection.

Method

Three defenses are proposed and empirically evaluated:

  1. Prompt-Level Defense Framework: Sequential pipeline of (i) sanitization/normalization removing URLs, Base64, and injection tokens; (ii) stochastic synonym paraphrasing to break adversarial token sequences; (iii) embedding-based risk scoring via cosine similarity to a known-unsafe prompt centroid, which selects an adaptive system prompt from mild guidance to strict refusal.

  2. Logit-Based Steering Defense: Training-free inference-time intervention. Safety-sensitive layers are auto-selected by computing cosine gaps between safe/unsafe prompt logits (safety dissimilarity) and alignment with a danger prior. Top-K salient tokens are clustered into a Refusal Set (R) and Danger Set (B); a steering vector v = (ΣEᵢ[R] − ΣEⱼ[B]) / ‖…‖₂ is injected into those layers, modulated by a sigmoid-gated dynamic risk score. This shifts activations toward refusal semantics without retraining.

  3. MetaGPT-Based Agent Defense: A three-agent pipeline—Rephrase Agent (sanitizes intent), Core LLM (generates response), Judge Agent (enforces safety/domain policy)—communicating via structured artifacts and a publish-subscribe message pool. Iterative corrective loops trigger when outputs violate domain constraints.

Evaluation uses XSTest and In-the-Wild Jailbreak Prompts on Llama-3.1-8B-Instruct (aligned) and dolphin-2.9.1-llama-3-8b (unaligned), with Gemma-3-4b-it as the LLM judge.

Key Contributions

  • Systematic taxonomy of jailbreak defenses across prompt-level, model-level, logit-based, agent-based, and training-time categories.
  • Logit-based attract–repel steering with automatic layer/token selection, operable without retraining.
  • Adaptation of MetaGPT as a domain-specific jailbreak defense via role specialization and multi-stage validation.
  • Empirical head-to-head comparison of all three proposed methods on the same benchmark/model pair.

Results

  • Prompt-Level Defense: Aligned model ASR 0.36 → 0.28 (−22%); Unaligned 0.60 → 0.55 (−8%).
  • Logit-Based Steering: Aligned ASR 0.385 → 0.315 (−18%); Unaligned 0.520 → 0.295 (−43%).
  • MetaGPT Agent Defense: Unaligned model ASR → 0.00 (full mitigation); Aligned model maintained low baseline (~0.33) and was fully secured.
  • Aligned model consistently outperforms unaligned across all defense conditions, confirming that training-time alignment compounds inference-time defenses.

Limitations

  • MetaGPT defense requires three sequential inference passes per query, making it computationally expensive and unsuitable for low-latency real-time applications.
  • Evaluation dataset is relatively small and constructed by merging/deduplicating two benchmarks; generalization to broader attack taxonomies is unclear.
  • Prompt-level defense shows only modest gain on the unaligned model (−8%), suggesting it is insufficient as a standalone defense for non-aligned models.
  • The LLM judge (Gemma-3-4b-it) for safety evaluation may introduce its own biases or errors in ASR measurement.
  • No ablation on the logit steering's layer-selection heuristic or sensitivity to the steering strength hyperparameter α.

Relevance to Agentic AI / LLM Agents

The MetaGPT-based defense directly addresses a core vulnerability in deployed LLM agents: topic deviation and jailbreak under prompt injection in multi-agent pipelines. The role-specialization and structured communication protocols (publish-subscribe, SOPs, iterative corrective loops) are architectural patterns increasingly adopted in production agentic systems (LangGraph, CrewAI), making the findings immediately applicable to agent framework design. For researchers building multi-agent systems, this work empirically validates that layered validation—Rephrase → Generate → Judge—can fully neutralize adversarial inputs while preserving task utility, at the cost of compute. The logit-steering approach is also relevant for agents needing lightweight, inference-time safety control without retraining the underlying model.