On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation

by   Nicolas Brosse, et al.
Ecole Polytechnique

Uncertainty quantification for deep learning is a challenging open problem. Bayesian statistics offer a mathematically grounded framework to reason about uncertainties; however, approximate posteriors for modern neural networks still require prohibitive computational costs. We propose a family of algorithms which split the classification task into two stages: representation learning and uncertainty estimation. We compare four specific instances, where uncertainty estimation is performed via either an ensemble of Stochastic Gradient Descent or Stochastic Gradient Langevin Dynamics snapshots, an ensemble of bootstrapped logistic regressions, or via a number of Monte Carlo Dropout passes. We evaluate their performance in terms of selective classification (risk-coverage), and their ability to detect out-of-distribution samples. Our experiments suggest there is limited value in adding multiple uncertainty layers to deep classifiers, and we observe that these simple methods strongly outperform a vanilla point-estimate SGD in some complex benchmarks like ImageNet.


Uncertainty Quantification in Deep Learning through Stochastic Maximum Principle

We develop a probabilistic machine learning method, which formulates a c...

Calibration and Uncertainty Quantification of Bayesian Convolutional Neural Networks for Geophysical Applications

Deep neural networks offer numerous potential applications across geosci...

Uncertainty estimation for out-of-distribution detection in computational histopathology

In computational histopathology algorithms now outperform humans on a ra...

What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers

When deployed for risk-sensitive tasks, deep neural networks must includ...

Reliable uncertainty estimate for antibiotic resistance classification with Stochastic Gradient Langevin Dynamics

Antibiotic resistance monitoring is of paramount importance in the face ...

Which models are innately best at uncertainty estimation?

Deep neural networks must be equipped with an uncertainty estimation mec...

Deep Classifiers with Label Noise Modeling and Distance Awareness

Uncertainty estimation in deep learning has recently emerged as a crucia...

Please sign up or login with your details

Forgot password? Click here to reset