Skip to content

VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VADTree is a training-free video anomaly detection framework that replaces fixed-length sliding temporal windows with a Hierarchical Granularity-aware Tree (HGTree) built from a pre-trained Generic Event Boundary Detection (GEBD) model. VLMs and LLMs then score anomalies node-wise across coarse and fine tree clusters, with intra-cluster refinement and inter-cluster fusion. It achieves state-of-the-art among training-free methods on UCF-Crime, XD-Violence, and MSAD.

Problem

Existing training-free VAD methods (e.g., LAVAD, VERA) segment video using fixed-length sliding temporal windows, which cannot align with anomalous events of varying durations (seconds to minutes). Fixed windows cause abrupt semantic discontinuities, conflation of irrelevant content, and amplified VLM hallucinations. Domain-specific temporal samplers (e.g., HolmesVAU) that could adapt require in-domain fine-tuning and generalize poorly across domains.

Method

  1. HGTree Construction: A pre-trained EfficientGEBD model generates frame-level boundary confidence scores over overlapping 10-second clips concatenated into a global sequence. A binary tree is built via depth-first recursive splitting at highest-confidence boundaries (threshold γ_min = 0.4). K-means on boundary confidences stratifies nodes into coarse (strong, clear boundaries → parent nodes) and fine (weaker, localized boundaries → child/leaf nodes), with RemoveDup pruning and Complete replicating childless leaf nodes.

  2. Prior-infused Node Scoring: An LLM generates multi-dimensional priors over scene, object, and action dimensions (excluding micro-expressions and audio cues). These priors are injected into a VLM (LLaVA-Video-7B-Qwen2) to caption each node's sampled frames. A second LLM (DeepSeek-R1-Distill-Qwen-14B with thinking mode) converts captions into discrete anomaly scores ∈ {0, 0.1, …, 1}.

  3. Intra-cluster Refinement: Initial node scores are refined by computing cosine similarity (via ImageBind features) among nodes within the same cluster; the refined score is a softmax-weighted average of the top-K most similar nodes' scores, suppressing VLM/LLM hallucination noise.

  4. Inter-cluster Correlation: For each coarse parent node with m fine children, intra-cluster cohesion w_i is measured as the variance of children's denoised scores. Low variance → parent (coarse) score dominates; high variance → children (fine) scores dominate. Final frame-wise scores blend parent and child scores weighted by β·ŵ_i.

Key Contributions

  • HGTree: a GEBD-driven adaptive binary tree that achieves mIoU of 0.52 (UCF-Crime) and 0.64 (XD-Violence) vs. 0.51/0.44 for a 10-second fixed window.
  • Prior-infused node scoring that injects scene/object/action priors into VLMs for anomaly-aware description generation.
  • Intra-cluster node refinement via semantic similarity to suppress per-node hallucinations.
  • Inter-cluster cohesion-driven fusion that dynamically balances coarse and fine granularity anomaly evidence.
  • Training-free framework that requires no domain-specific data or parameter updates.

Results

  • UCF-Crime (AUC ROC): 84.74% — surpasses LAVAD (80.28%) by +4.5%, EventVAD (82.03%) by +2.7%, SUVAD (83.90%) by +0.8%; outperforms all one-class and unsupervised methods.
  • XD-Violence (AUC ROC / AP): 90.44% / 67.82% — exceeds LAVAD (87.51% AUC) by +3.0% and EventVAD (90.55% / 64.04%) with higher AP (+3.8%); VADTree* (with audio captions via Kimi-Audio-7B-Instruct) reaches 90.55% AUC / 68.85% AP.
  • MSAD: 89.32% AUC — surpasses all weakly-supervised baselines including Ï€-VAD (88.68%) on overall AUC.
  • Ablation (UCF-Crime): HGTree fine cluster alone = 71.57%; +prior scoring = 75.67%; +intra-cluster refinement = 83.05%; +inter-cluster correlation = 84.74%.
  • HGTree with two-cluster hierarchy outperforms single-cluster (leaf-only) by +1.9% AUC.

Limitations

  • Inference overhead: DeepSeek-R1-Distill-Qwen-14B with thinking mode adds significant latency; no runtime benchmarks are reported.
  • Two-granularity (coarse/fine) hierarchy is fixed by design; dynamic depth selection is not explored.
  • GEBD model processes only short clips (10s window), requiring overlapping stitching for long videos, which may produce boundary artifacts at seams.
  • Audio modality requires a separate model (Kimi-Audio-7B-Instruct); the baseline VADTree is visual-only and lags audio-augmented VADTree* on XD-Violence.
  • Coarse/fine score divergence due to lack of a unified inference standard across nodes is acknowledged but only partially mitigated by inter-cluster fusion.
  • Ï€-VAD retains an advantage on anomaly-specific sub-metrics (AUCa, APa) on MSAD due to supervised fine-tuning on anomalous segments.

Relevance to Vision-Language Models

VADTree exemplifies an emerging paradigm in which VLMs serve as zero-shot visual describers within structured temporal pipelines rather than monolithic video-understanding models. The paper demonstrates that VLM hallucinations are a practical bottleneck in long-video reasoning tasks and proposes intra-cluster semantic similarity refinement as a targeted mitigation—directly relevant to anyone studying VLM reliability in temporal grounding. The prior-injection mechanism (LLM-generated scene/object/action priors fed as VLM prompts) is a reusable pattern for steering frozen VLMs toward domain-relevant perception without fine-tuning. The finding that GEBD-structured segmentation substantially outperforms fixed temporal windows is a concrete pointer for VLM-based video understanding research more broadly.