Skip to content

Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

🕒 Published (v1): 2026-07-08 20:42 UTC · Source: Arxiv · Venue: ACL · link

Why this paper was selected

ACL survey; comprehensive multimodal unlearning across modalities; timely safety/privacy topic

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This survey provides a unified, system-first taxonomy of machine unlearning methods across vision-language models (VLMs), diffusion models (DMs), large language models (LLMs), and audio foundation models (AFMs). It organizes methods by intervention stage and control pathway rather than by optimization objective, enabling cross-modal comparison. The work identifies open challenges in evaluation, adversarial robustness, and deployment.

Problem

Multimodal foundation models trained on web-scale data inadvertently memorize sensitive, copyrighted, biased, or unsafe cross-modal associations. Retraining from scratch after deletion requests is impractical, and targeted removal is difficult because knowledge is distributed across shared representations spanning multiple modalities. Prior surveys are either unimodal or adopt algorithm-centric taxonomies that obscure deployment-relevant intervention points.

Method

The survey adopts a system-first taxonomy that splits methods by (1) forgetting target scope—instance-level vs. concept-level—and (2) intervention stage and control pathway within the multimodal pipeline. Five intervention stages are identified:

  1. Data-side interventions: input perturbation (constrained \(\|p_\text{img}(x)\| \le \epsilon_\text{img}\), \(\|p_\text{txt}(t)\| \le \epsilon_\text{txt}\)) and data hygiene/prompt normalization.
  2. Training-time edits: direct gradient methods, constrained risk minimization (\(\min_\theta \mathcal{J}_R(\theta) + \Omega(\theta,\theta_0)\) s.t. \(\mathcal{C}_f(\theta)\le0\)), mask-driven selective unlearning via saliency/Fisher masks, and distillation-based transfer.
  3. Architecture-constrained: pruning, freezing, or layer-scoped updates.
  4. Training-free: weight-space linear unlearning (task vector subtraction \(\theta' = \theta_0 - \Delta\tau\)), and representation projection unlearning \(h' = (I - UU^\top)h\) where \(U\) spans the forget subspace.
  5. Decoding-time: guidance-path and conditioning-path control that reshapes the denoising score without weight changes.

The unlearning objective is formalized as \(\min_\theta F_\text{forget}(\theta; D_f) + \lambda F_\text{retain}(\theta; D_r)\), with approximate retraining equivalence guaranteed by an \((\varepsilon,\delta)\) criterion mirroring differential privacy.

Key Contributions

  • Unified formalization of multimodal unlearning across image, text, video, and audio, including modality-specific objectives for VLMs and DMs.
  • System-first taxonomy organizing ~40+ methods by intervention stage and control pathway, enabling cross-architecture comparison.
  • Comprehensive survey of datasets (CelebA, CelebA-HQ, VoxCeleb1, UCF101, etc.) and benchmarks (MU-Bench, MLLMU-Bench, MMUBench, UnlearnCanvas, UnSLU-BENCH, etc.) across modalities.
  • Identification of open challenges: evaluation gaps, adversarial robustness, and deployment constraints for accountable unlearning.
  • Curated public repository of methods and resources.

Results

This is a survey paper; no new empirical results are reported. The paper catalogues existing benchmarks and their evaluation criteria: - MU-Bench: 9 datasets, 20 architectures, evaluating efficacy, utility, and efficiency. - MLLMU-Bench: 500 fictitious + 153 real celebrities, 20.7K QA pairs for privacy unlearning. - PEBench: 200 identities, 8K images, 16K QA pairs for synthetic identity/event unlearning. - UnlearnCanvas: 60 styles, 20 objects for artistic style forgetting in DMs. - Six-CD: 6 concept categories with dual-version prompts for cross-category concept suppression. - SafeEraser: 3K images, 28.8K QA pairs for harmful knowledge removal in VLMs.

Limitations

  • No new experimental results; claims about method effectiveness rely entirely on reported numbers from cited work, which may use inconsistent evaluation protocols.
  • The taxonomy covers methods through mid-2025/early 2026; rapidly emerging work may not be fully represented.
  • Theoretical guarantees (the \((\varepsilon,\delta)\) criterion) are not empirically validated in practice for most covered methods.
  • Audio and video unlearning coverage is narrower than vision-language coverage, reflecting the state of the literature rather than a design choice.
  • Adversarial robustness of unlearning methods (resistance to membership inference, extraction attacks) is identified as an open problem but not systematically evaluated here.

Relevance to Vision-Language Models

VLMs are a primary focus of this survey: the formalization explicitly targets vision encoders \(f_v\), text encoders \(f_t\), and fusion heads \(F\), with VLM-specific objectives using concept hinge losses (\(S_\theta(x, c_f) \le m\)) and saliency-masked gradient updates. The taxonomy directly addresses how to selectively remove multimodal associations—e.g., private faces linked to text identifiers, harmful visual-textual concepts—without degrading cross-modal representations, which is a central challenge in deploying VLMs under privacy regulations (GDPR right-to-be-forgotten). Researchers working on VLM alignment, safety, or data governance will find the benchmark comparison (MMUBench, MLLMU-Bench, SafeEraser, UnLOK-VQA) and the training-free RPU/WLU methods particularly actionable, as they enable post-hoc correction without retraining large vision-language checkpoints.