Further Analysis of Outlier Detection with Deep Generative Models

by   Ziyu Wang, et al.

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling. In this work, we present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. From this vantage point we propose a novel outlier test, the empirical success of which suggests that the failure of existing likelihood-based outlier tests does not necessarily imply that the corresponding generative model is uncalibrated. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers. In aggregate, these results suggest that modifications to the standard evaluation practices and benchmarks commonly applied in the literature are needed.


page 6

page 12

page 17


Shaken, and Stirred: Long-Range Dependencies Enable Robust Outlier Detection with PixelCNN++

Reliable outlier detection is critical for real-world applications of de...

Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models

Out-of-distribution (OOD) detection is an important task in machine lear...

Understanding Failures in Out-of-Distribution Detection with Deep Generative Models

Deep generative models (DGMs) seem a natural fit for detecting out-of-di...

Entropic Issues in Likelihood-Based OOD Detection

Deep generative models trained by maximum likelihood remain very popular...

Outlier Detection using Generative Models with Theoretical Performance Guarantees

This paper considers the problem of recovering signals from compressed m...

Efficient remedies for outlier detection with variational autoencoders

Deep networks often make confident, yet incorrect, predictions when test...

Evaluation of Sampling Methods for Scatterplots

Given a scatterplot with tens of thousands of points or even more, a nat...