Skip to content

Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LogitGap is a training-free, post-hoc OOD detection scoring function that replaces MaxLogit and MCM by explicitly computing the average gap between the maximum logit and remaining logits. It exploits the empirical observation that ID samples produce sharper, more peaked logit distributions while OOD samples produce flatter ones. LogitGap consistently achieves state-of-the-art OOD detection across CLIP-based and traditional vision models.

Problem

Existing post-hoc OOD scoring functions underutilize logit-space information: MaxLogit ignores non-maximum logits entirely, while MCM's softmax normalization discards absolute logit magnitudes, causing different logit patterns to collapse to similar probability distributions. Neither fully exploits the distributional contrast between ID and OOD logits.

Method

LogitGap sorts the K predicted logits in descending order and computes the average gap between the top logit and all remaining logits:

S = (1/(K−1)) * Σ_{j=2}^{K} (z'_1 − z'_j)

This is equivalent to z'_1 − mean(z'_2, ..., z'_K), directly measuring logit peakedness. A theoretical result (Theorem 4.1) shows that when temperature τ > 2(K−1), LogitGap's FPR is provably lower than MCM's.

LogitGap-topN refines this by restricting computation to the top-N logits (default N = 20% of K), excluding noisy tail logits that overlap heavily between ID and OOD. The optimal N is selected via a data-efficient criterion: given ~100 ID samples and synthetic OOD (interpolation + noise injection), choose N that maximizes the mean score gap between ID and synthetic OOD distributions.

Key Contributions

  • Novel post-hoc OOD scoring function that exploits the full logit margin (max − mean of remaining) rather than max alone or softmax-normalized max.
  • Theoretical proof that LogitGap's FPR is upper-bounded by MCM's FPR under a mild temperature condition.
  • Training-free, data-efficient hyperparameter selection for N using only ~100 ID samples and synthetic OOD.
  • Demonstrated complementarity with training-based methods (OE, MixOE, ID-Like, CoOp).

Results

  • Zero-shot on ImageNet (CLIP ViT-B/16): LogitGap reduces FPR95 by 3.65% vs. MCM; LogitGap* by additional margin.
  • Zero-shot on ImageNet-100 (CLIP ViT-B/16): FPR95 improvement of 5.78% over MCM.
  • Semantically hard OOD (ImageNet-10 vs. ImageNet-20): LogitGap reduces FPR95 by 8.40% vs. MCM in the reverse split.
  • Few-shot (1-shot, ImageNet ID): LogitGap* + ID-Like improves FPR95 by 7.39% and AUROC by 7.08% vs. ID-Like alone.
  • Traditional setting (ResNet-50, CIFAR-10 ID, OpenOOD protocol): LogitGap avg. FPR95 = 34.91% vs. 37.17% for MSP; also reduces MixOE's avg. FPR95 by 8.29%.
  • Ablations show nonlinear variants (exp, square, sqrt) perform comparably to the linear formulation.

Limitations

  • Primarily designed for visual classification models; extension to text or speech OOD detection is left as future work.
  • The top-N selection strategy relies on synthetic OOD data quality; distribution mismatch between synthetic and real OOD could affect N choice.
  • Evaluated only on CLIP (ViT-B/16, ViT-L/14) and ResNet-50; generalization to other VLM architectures (e.g., LLaVA, Flamingo) is not demonstrated.
  • The theoretical guarantee (Ï„ > 2(K−1)) may be difficult to satisfy in practice for large K classification tasks.

Relevance to Vision-Language Models

LogitGap is directly applicable to CLIP-style VLMs, where the logit vector is produced by cosine similarity between image embeddings and text class embeddings — the primary zero-shot classification pathway. It outperforms MCM, the dominant CLIP OOD baseline, without requiring prompt engineering or auxiliary data, making it a lightweight drop-in improvement for any downstream VLM deployment that needs open-world reliability. The peakedness-of-logits insight is architecturally agnostic and likely transfers to other VLMs with discrete class-logit outputs (e.g., ALIGN, SigLIP). This work sits at the intersection of VLM robustness and deployment safety, relevant to anyone applying CLIP or similar models in open-world settings.