Temporal Probability Calibration

02/07/2020
by   Tim Leathart, et al.
0

In many applications, accurate class probability estimates are required, but many types of models produce poor quality probability estimates despite achieving acceptable classification accuracy. Even though probability calibration has been a hot topic of research in recent times, the majority of this has investigated non-sequential data. In this paper, we consider calibrating models that produce class probability estimates from sequences of data, focusing on the case where predictions are obtained from incomplete sequences. We show that traditional calibration techniques are not sufficiently expressive for this task, and propose methods that adapt calibration schemes depending on the length of an input sequence. Experimental evaluation shows that the proposed methods are often substantially more effective at calibrating probability estimates from modern sequential architectures for incomplete sequences across a range of application domains.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

01/30/2020

Better Multi-class Probability Estimates for Small Data Sets

Many classification applications require accurate probability estimates ...
07/31/2018

Probability Calibration Trees

Obtaining accurate and well calibrated probability estimates from classi...
07/03/2020

Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain

Label disagreement between human experts is a common issue in the medica...
02/24/2020

Better Classifier Calibration for Small Data Sets

Classifier calibration does not always go hand in hand with the classifi...
05/26/2020

Improving Regression Uncertainty Estimates with an Empirical Prior

While machine learning models capable of producing uncertainty estimates...
05/01/2018

Performance Analysis of Distributed Radio Interferometric Calibration

Distributed calibration based on consensus optimization is a computation...
10/15/2018

Computer model calibration based on image warping metrics: an application for sea ice deformation

Arctic sea ice plays an important role in the global climate. Sea ice mo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.