Likelihood Ratios for Out-of-Distribution Detection

06/07/2019
by   Jie Ren, et al.
6

Discriminative neural networks offer little or no performance guarantees when deployed on data not generated by the same process as the training distribution. On such out-of-distribution (OOD) inputs, the prediction may not only be erroneous, but confidently so, limiting the safe deployment of classifiers in real-world applications. One such challenging application is bacteria identification based on genomic sequences, which holds the promise of early detection of diseases, but requires a model that can output low confidence predictions on OOD genomic sequences from new bacteria that were not present in the training data. We introduce a genomics dataset for OOD detection that allows other researchers to benchmark progress on this important problem. We investigate deep generative model based approaches for OOD detection and observe that the likelihood score is heavily affected by population level background statistics. We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics. We benchmark the OOD detection performance of the proposed method against existing approaches on the genomics dataset and show that our method achieves state-of-the-art performance. We demonstrate the generality of the proposed method by showing that it significantly improves OOD detection when applied to deep generative models of images.

READ FULL TEXT

page 7

page 15

research
08/18/2022

Out-of-distribution Detection via Frequency-regularized Generative Models

Modern deep generative models can assign high likelihood to inputs drawn...
research
02/23/2022

Better Modelling Out-of-Distribution Regression on Distributed Acoustic Sensor Data Using Anchored Hidden State Mixup

Generalizing the application of machine learning models to situations wh...
research
12/30/2019

Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog

The task of identifying out-of-domain (OOD) input examples directly at t...
research
08/29/2022

Shaken, and Stirred: Long-Range Dependencies Enable Robust Outlier Detection with PixelCNN++

Reliable outlier detection is critical for real-world applications of de...
research
05/31/2018

Cyberattack Detection using Deep Generative Models with Variational Inference

Recent years have witnessed a rise in the frequency and intensity of cyb...
research
05/26/2023

SR-OOD: Out-of-Distribution Detection via Sample Repairing

It is widely reported that deep generative models can classify out-of-di...
research
12/08/2022

Transformer-based normative modelling for anomaly detection of early schizophrenia

Despite the impact of psychiatric disorders on clinical health, early-st...

Please sign up or login with your details

Forgot password? Click here to reset