Skip to content

ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ImageSentinel is a framework that protects private visual datasets from unauthorized use in Retrieval-Augmented Image Generation (RAIG) systems. It synthesizes "sentinel images" — visually consistent decoy images embedding random character sequences — that act as traceable verification keys detectable in RAIG outputs. Traditional digital watermarking fails here because RAIG's feature extraction/recombination destroys embedded signals; sentinel images bypass this by exploiting the retrieval mechanism itself.

Problem

No effective mechanism existed to detect unauthorized inclusion of private image datasets in RAIG reference databases. Digital watermarking is unsuitable because image generation processes destroy watermark signals, unlike text-based RAG where content is paraphrased rather than feature-recombined. Semantic-based retrieval approaches also fail in large visual databases due to semantic collision (multiple images match the same description) and RAIG systems that bypass retrieval entirely when internal generators suffice.

Method

ImageSentinel operates in two phases:

Protection phase: 1. Key generation: random 6-character alphanumeric strings (e.g., "VasWiW") are generated — strings highly unlikely to appear in real user prompts. 2. Sentinel image synthesis: GPT-4o extracts comprehensive semantic attributes (subject, style, color, composition) from randomly selected reference images in the private dataset via VLM-based attribute extraction. A structured text-to-image prompt then embeds both the semantic description and the key character string visually into a new sentinel image using GPT-4o's text-to-image capability. Sentinel images are added to (not replacing) the private dataset: D̂p = Dp ∪ Ds.

Detection phase: The dataset owner queries a suspected RAIG system with prompts containing the key strings (e.g., "A picture of 'VasWiW'."). If the system has ingested the protected dataset, the retriever fetches the matching sentinel image as a reference, and the generated output will visually resemble it. Detection score is computed as average DINO cosine similarity between generated images and corresponding sentinel images across all keys; if it exceeds threshold η, unauthorized use is flagged.

Key Contributions

  • Formalizes the novel problem of protecting visual datasets from unauthorized RAIG usage, distinct from prior individual-image or model watermarking.
  • Proposes ImageSentinel: sentinel-image-based protection that exploits retrieval mechanisms rather than watermark persistence.
  • Introduces random character sequences as retrieval keys, achieving 100% triggering rate vs. 21.3–39.0% for semantic-based retrieval (SDXL/OmniGen, CLIP retriever).
  • Demonstrates that adding sentinel images (not replacing originals) preserves authorized generation quality with only marginal CLIP/DINO/SigLIP score degradation.

Results

  • Detection AUC (LLaVA-Pretrain, 10 queries): ImageSentinel achieves 1.000 on SDXL and GPT-4o, 0.989 on OmniGen; Ward-HiDDeN stays at 0.585/0.600/0.555 and Ward-FIN at 0.559/0.538/0.531.
  • TPR@1%FPR (20 queries, SDXL): ImageSentinel 1.000 vs. Ward-HiDDeN 0.074, Ward-FIN 0.013.
  • Product-10K (30k images, 8 queries): AUC 1.000, TPR@1%FPR 1.000, TPR@10%FPR 1.000.
  • Retrieval accuracy (Hit@1): ImageSentinel 69.7% vs. semantic-based 58.3% (CLIP retriever).
  • Triggering rate: 100% (ImageSentinel) vs. 21.3% (semantic, SDXL) and 39.0% (semantic, OmniGen).
  • Generation quality (SDXL, CLIP↑): Original RAIG 0.776, ImageSentinel 0.772 vs. sentinel-replacement 0.708.
  • Optimal key length: 6 characters (AUC 0.997, T@1%F 0.980) outperforms 4- and 8-character keys.
  • Adaptive detect-and-inpaint attack (50 queries): AUC recovers to 0.91 but drops to 0.62 at 5 queries; generation quality degrades minimally (CLIP 0.769 vs. 0.772).

Limitations

  • Relies on the text-to-image model's ability to embed legible characters; SDXL fails at this, limiting the approach to powerful models like GPT-4o.
  • Detect-and-inpaint adaptive attack (EasyOCR + SD 2.0 Inpainting) significantly reduces detection at low query counts (AUC 0.62 at 5 queries), requiring many more queries (~50–100) to recover robustness.
  • The approach is inherently text-in-image based; if a RAIG system's retriever is not susceptible to character-string triggers, detection fails.
  • DINO cosine similarity as the detection metric may not be optimal; more precise similarity functions are left as future work.
  • No evaluation of adversaries who pre-filter or detect sentinel images before database ingestion beyond the single inpainting attack.

Relevance to Vision-Language Models

VLMs (specifically GPT-4o) are central to ImageSentinel in two ways: as the attribute-extraction backbone that enables stealthy, semantically consistent sentinel synthesis, and as the text-to-image generation engine that reliably embeds character keys into images. This paper illustrates a security-adjacent application of VLMs in dataset provenance and IP protection — a growing concern as RAIG systems proliferate and depend on curated visual corpora. It also reveals that CLIP/SigLIP-based retrievers (standard VLM encoders) can be hijacked by crafted out-of-distribution text strings, which has implications for understanding retrieval robustness in VLM-powered systems more broadly.