Layer Adaptive Deep Neural Networks for Out-of-distribution Detection

03/01/2022
by   Haoliang Wang, et al.
11

During the forward pass of Deep Neural Networks (DNNs), inputs gradually transformed from low-level features to high-level conceptual labels. While features at different layers could summarize the important factors of the inputs at varying levels, modern out-of-distribution (OOD) detection methods mostly focus on utilizing their ending layer features. In this paper, we proposed a novel layer-adaptive OOD detection framework (LA-OOD) for DNNs that can fully utilize the intermediate layers' outputs. Specifically, instead of training a unified OOD detector at a fixed ending layer, we train multiple One-Class SVM OOD detectors simultaneously at the intermediate layers to exploit the full spectrum characteristics encoded at varying depths of DNNs. We develop a simple yet effective layer-adaptive policy to identify the best layer for detecting each potential OOD example. LA-OOD can be applied to any existing DNNs and does not require access to OOD samples during the training. Using three DNNs of varying depth and architectures, our experiments demonstrate that LA-OOD is robust against OODs of varying complexity and can outperform state-of-the-art competitors by a large margin on some real-world datasets.

READ FULL TEXT
research
07/07/2020

SpinalNet: Deep Neural Network with Gradual Input

Over the past few years, deep neural networks (DNNs) have garnered remar...
research
04/30/2021

MOOD: Multi-level Out-of-distribution Detection

Out-of-distribution (OOD) detection is essential to prevent anomalous in...
research
05/03/2023

Morphological Classification of Galaxies Using SpinalNet

Deep neural networks (DNNs) with a step-by-step introduction of inputs, ...
research
08/16/2020

GLOD: Gaussian Likelihood Out of Distribution Detector

Discriminative deep neural networks (DNNs) do well at classifying input ...
research
05/24/2019

Not All Features Are Equal: Feature Leveling Deep Neural Networks for Better Interpretation

Self-explaining models are models that reveal decision making parameters...
research
08/04/2019

Theme Aware Aesthetic Distribution Prediction with Full Resolution Photos

Aesthetic quality assessment (AQA) of photos is a challenging task due t...
research
03/25/2021

Recent Advances in Large Margin Learning

This paper serves as a survey of recent advances in large margin trainin...

Please sign up or login with your details

Forgot password? Click here to reset