Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration

02/24/2021
by   Christian Tomani, et al.
0

We address the problem of uncertainty calibration and introduce a novel calibration method, Parametrized Temperature Scaling (PTS). Standard deep neural networks typically yield uncalibrated predictions, which can be transformed into calibrated confidence scores using post-hoc calibration methods. In this contribution, we demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We generalize temperature scaling by computing prediction-specific temperatures, parameterized by a neural network. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/20/2020

Post-hoc Uncertainty Calibration for Domain Drift Scenarios

We address the problem of uncertainty calibration. While standard deep n...
03/15/2020

Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks

Predicting calibrated confidence scores for multi-class deep networks is...
07/30/2021

Soft Calibration Objectives for Neural Networks

Optimal decision making requires that classifiers produce uncertainty es...
03/16/2020

Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning

This paper studies the problem of post-hoc calibration of machine learni...
06/25/2021

Improving Uncertainty Calibration of Deep Neural Networks via Truth Discovery and Geometric Optimization

Deep Neural Networks (DNNs), despite their tremendous success in recent ...
02/21/2020

Calibrating Deep Neural Networks using Focal Loss

Miscalibration – a mismatch between a model's confidence and its correct...
08/12/2020

Local Temperature Scaling for Probability Calibration

For semantic segmentation, label probabilities are often uncalibrated as...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.