On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation

01/22/2020
by   Nicolas Brosse, et al.
60

Uncertainty quantification for deep learning is a challenging open problem. Bayesian statistics offer a mathematically grounded framework to reason about uncertainties; however, approximate posteriors for modern neural networks still require prohibitive computational costs. We propose a family of algorithms which split the classification task into two stages: representation learning and uncertainty estimation. We compare four specific instances, where uncertainty estimation is performed via either an ensemble of Stochastic Gradient Descent or Stochastic Gradient Langevin Dynamics snapshots, an ensemble of bootstrapped logistic regressions, or via a number of Monte Carlo Dropout passes. We evaluate their performance in terms of selective classification (risk-coverage), and their ability to detect out-of-distribution samples. Our experiments suggest there is limited value in adding multiple uncertainty layers to deep classifiers, and we observe that these simple methods strongly outperform a vanilla point-estimate SGD in some complex benchmarks like ImageNet.

READ FULL TEXT
research
11/28/2020

Uncertainty Quantification in Deep Learning through Stochastic Maximum Principle

We develop a probabilistic machine learning method, which formulates a c...
research
05/25/2021

Calibration and Uncertainty Quantification of Bayesian Convolutional Neural Networks for Geophysical Applications

Deep neural networks offer numerous potential applications across geosci...
research
10/18/2022

Uncertainty estimation for out-of-distribution detection in computational histopathology

In computational histopathology algorithms now outperform humans on a ra...
research
02/23/2023

What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers

When deployed for risk-sensitive tasks, deep neural networks must includ...
research
11/27/2018

Reliable uncertainty estimate for antibiotic resistance classification with Stochastic Gradient Langevin Dynamics

Antibiotic resistance monitoring is of paramount importance in the face ...
research
06/05/2022

Which models are innately best at uncertainty estimation?

Deep neural networks must be equipped with an uncertainty estimation mec...
research
10/06/2021

Deep Classifiers with Label Noise Modeling and Distance Awareness

Uncertainty estimation in deep learning has recently emerged as a crucia...

Please sign up or login with your details

Forgot password? Click here to reset