Skip to content

Value-Aligned Prompt Moderation via Zero-Shot Agentic Rewriting for Safe Image Generation

🕒 Published (v1): 2025-11-12 09:52 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VALOR (Value-Aligned LLM-Overseen Rewriter) is a modular, zero-shot agentic pipeline that wraps any text-to-image (T2I) generator with a three-stage moderation harness: multi-granular risk detection, LLM-guided prompt rewriting, and post-generation safety verification with optional style-guided regeneration. It reduces unsafe T2I outputs by up to 100% while preserving CLIP similarity and image quality, without retraining the underlying generator.

Problem

Existing T2I safety defenses fall into static keyword filters (bypassed by paraphrasing), computationally expensive image-level classifiers, or fine-tuning approaches that degrade generation quality and generalize poorly. None address the three-way misalignment between text modality, visual modality, and human values—particularly (1) T2I models' inability to represent negation/prohibition, (2) cross-modal drift where benign words yield unsafe images (e.g., "zombie"), and (3) context-dependent value violations (e.g., pole dancing in a congressional setting).

Method

VALOR wraps a frozen T2I model \(G: \mathcal{P} \to \mathcal{I}\) with a moderation function \(F: \mathcal{P} \to \mathcal{P}_\text{safe}\) defined as:

\[F(p) = \begin{cases} p, & \text{if } \text{Is\_safe}(p) = \text{True} \land \text{Intent}(p) = \text{None} \\ \text{LLMRewrite}(p, c), & c = \text{Intent}(p) \end{cases}\]

Multi-granular detector \(\text{Is\_safe}(p) = \neg(W(p) \lor S(p) \lor V(p))\): - Word-level \(W(p)\): blocked-keyword set lookup. - Semantic-level \(S(p)\): sentence encoder cosine similarity against a reference unsafe-phrase set, flagged if \(\max_{r \in U} \cos(f(p), f(r)) > \tau_s\). - Value-level \(V(p)\): dual cosine similarity against sensitive-location set \(L\) and inappropriate-act set \(A\); flagged if both exceed threshold \(\tau_v\).

Intention disambiguator \(I(p)\): SpaCy dependency parse extracts (verb, negation/prohibition modifier, unsafe-object) triples; flags prompts where a constraint cue syntactically governs an unsafe visual concept—catching cases like "naked running is forbidden."

LLM rewriter: assigns risk category \(c \in \{\text{NSFW}, \text{VALUE}, \text{INTENTION}\}\) and invokes a chat LLM with a category-specific system prompt \(S_c\) to produce \(p' = \text{LLM}(\text{Template}(S_c, p))\).

Safety-guided regeneration: if the generated image \(I = G(p')\) fails a downstream safety checker \(C(I) = 0\), a domain suffix \(\delta\) (e.g., "in illustration style") is appended: \(\tilde{p} = p' + \delta\), nudging style without altering semantics.

Key Contributions

  • First zero-shot prompt moderation agent for T2I generation requiring no model retraining or domain-specific fine-tuning.
  • Three-dimensional misalignment taxonomy: single-modal, cross-modal, and machine–human.
  • Two new datasets: Intent-452 (intention ambiguity, 60.84% ambiguous, 22.57% blocked) and Value-412 (value alignment, 62.86% unsafe).
  • Modular pipeline composable across any chat LLM (Deepseek-7b, Qwen-1.8B, Zephyr-7b, Llama-3.2-3B) and any T2I backbone (SDV1.4, SDXL, SDV3.5, PixArt-α).

Results

  • vs. all baselines on 4chan: VALOR SAFE = 99.6%, NRR-N = 97.0%, NRR-Q = 99.2%, CLIP = 20.52—highest across all metrics.
  • vs. all baselines on I2P-Sexual: SAFE = 92.7%, NRR-N = 94.8%, NRR-Q = 98.5%, CLIP = 27.69 (best), FID = 15.89 (lowest), LPIPS = 0.8289 (lowest).
  • Adversarial jailbreaking (SneakyPrompt, QF-Attack, MMP-Attack, MMA-Diffusion): NRR 98.0%–100.0%; best baseline (SLD-Max) reaches only 97.0% on I2P-Illegal.
  • Intent-452 detection: ACC 92.0%–100.0%; FPR/FNR below 10% for all models except Qwen-1.8B (FNR up to 33.3%).
  • Value-412 detection: ACC 86.2%–98.7%; SAFE ≥ 84.9% across all T2I backbones.
  • Latency overhead: 0.575–2.54 s per prompt for LLM rewriting step (acceptable per authors).

Limitations

  • Qwen-1.5-1.8B-Chat exhibits FNR up to 33.3% on Intent-452, showing strong sensitivity to LLM capability; smaller models degrade safety recall substantially.
  • Safety-guided regeneration is passive and cannot substitute proactive value alignment; if the rewriter fails, style suffixes may dilute but not eliminate unsafe semantics.
  • Keyword blocklist \(B\) and reference unsafe-phrase set \(U\) are static; adversarially crafted prompts outside these distributions may evade word- and semantic-level detection.
  • Ablation is only reported for one (Deepseek + SDV1.4) configuration; cross-model ablation coverage is incomplete in the provided text.
  • Time overhead scales with LLM size and T2I model; the full pipeline adds up to ~37 s/prompt on SDV3.5 (total latency, not just overhead).

Relevance to Harnesses / Meta-Harnesses

VALOR is a textbook meta-harness: it wraps an unmodified generative model with a layered agentic pipeline—detector → LLM rewriter → verifier → optional regenerator—that orchestrates multiple specialized modules under a unified control flow without touching the underlying model's weights. The conditional routing logic (Eq. 2) and post-generation feedback loop (Eq. 3/12) mirror the detect-act-verify pattern common to safety and reliability harnesses. For researchers building meta-harnesses, VALOR illustrates how category-specific system prompts can substitute for fine-tuned modules, and how a lightweight suffix-augmentation step can serve as a fallback when the primary rewriting stage fails—a reusable design pattern for any harness that must satisfy both a safety oracle and a quality objective.