Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper proposes KCMP (Knowledge-Calibrated Memory Probing), the first black-box membership inference attack (MIA) framework for large vision-language models (LVLMs). It determines whether a suspected image was in an LVLM's training set by querying the model with semantically confounding masked-region prediction tasks and measuring prediction accuracy—without any access to logits or internal features. KCMP substantially outperforms existing black-box baselines and approaches gray-box performance as the number of aggregated samples grows.

Problem

Existing MIA methods for LVLMs assume white- or gray-box access—specifically, token-level log-probabilities or output logits—to compute likelihood-based membership scores (e.g., Min-K%, MaxRényi-K%). Real-world LVLM APIs (e.g., GPT-4o) expose only generated text, making these methods inapplicable. The fundamental difficulty in the purely black-box regime is disentangling memorization (model has seen this specific image) from generalization (model can answer correctly from broad world knowledge).

Method

KCMP operates in three stages:

1. Semantic Mask Prediction Task Construction. Given a suspected image, SAM segments primary objects. For each object, two probe types are constructed:
2. Shape probe: the object is masked with a black patch; GPT-4o generates K plausible but incorrect alternative objects as distractors; the LVLM must select the correct object from the shuffled set.

3. Color probe: the image is converted to grayscale with a bounding box around the target object; GPT-4o generates plausible but absent colors as distractors; the LVLM must identify the original color.

4. Prior Knowledge Calibration. Each probe is scored with a filter score f = Avg(u, r), where u is CLIP cosine similarity between the object patch and a text description of the image (relevance), and r is a rationality score from a reasoning LLM (GPT-o1) that estimates whether the task is solvable via general knowledge alone. Probes where distractors are clearly impossible or the answer is obvious are discarded; only the top-N scoring probes are retained.

5. Instruction-based Confidence Evaluation. The black-box target LVLM is queried R times on each retained probe; binary prediction accuracy is recorded. The final membership score t^x is the mean accuracy across all retained probes. High t^x indicates likely training membership.

Key Contributions

• First black-box MIA framework for LVLMs, requiring only generated text outputs.
• Shape-and-color dual-probe construction that transforms membership detection into a multiple-choice visual question-answering task.
• Prior knowledge calibration via CLIP relevance + LLM rationality scoring to filter out tasks resolvable without memorization.
• A new IID-controlled MIA benchmark built from the DAM model (200 member / 200 non-member images from COCOStuff, same distribution).

Results

• VL-MIA/Flickr, K=10 (set size): KCMP average AUC 0.794 vs. Image Infer (black-box baseline) 0.514; MaxRényi-K% (gray-box) 0.848.
• VL-MIA/DALL-E, K=10: KCMP avg AUC 0.700 vs. Image Infer 0.511; MaxRényi-K% 0.732.
• Scaling with K: KCMP improves steadily; at K=50 on LLaVA/Flickr, AUC reaches 97.9 (vs. MaxRényi-K% 100.0); on LLaMA Adapter, KCMP surpasses MaxRényi-K% at K=30 and K=40 (e.g., KCMP 95.7 vs. MaxRényi-K% 75.7 at K=30).
• DAM benchmark (sample-level): KCMP AUC 0.584 vs. Image Infer Rouge 0.502; below Min-K% (0.743) but above Max-Prob-Gap (0.601).
• Ablations: Combining shape + color probes outperforms either alone; filtering improves AUC across all models; performance saturates at N=12 top probes per image.

Limitations

• Requires semantically rich natural images with visually distinct objects; performance degrades for medical scans, document images, or domains with low semantic diversity where shape/color probes are uninformative.
• Relies on external commercial APIs (GPT-4o for distractor generation, GPT-o1 for rationality scoring, SAM for segmentation), introducing inference cost and latency.
• At small K (few samples), KCMP lags gray-box methods meaningfully; effectiveness is primarily demonstrated at the dataset/set level via aggregation.
• Detection score is binary accuracy over multiple-choice questions, which may be noisy for single samples.

Relevance to Vision-Language Models

KCMP directly targets the memorization behavior of LVLMs—a property that emerges from their large parameter count and training on massive image-text corpora—and exposes a real privacy risk in deployed systems like GPT-4o. For researchers tracking VLMs, this work provides a principled audit tool applicable to closed-source models, advancing understanding of what fine-grained visual content (object identity, color) gets memorized versus generalized during multimodal training. The probe design also implicitly reveals how cross-modal alignment (CLIP similarity between objects and text descriptions) correlates with what the model memorizes, which is relevant to work on LVLM training dynamics and data governance.