DeepAI AI Chat
Log In Sign Up

DEUP: Direct Epistemic Uncertainty Prediction

02/16/2021
by   Moksh Jain, et al.
9

Epistemic uncertainty is the part of out-of-sample prediction error due to the lack of knowledge of the learner. Whereas previous work was focusing on model variance, we propose a principled approach for directly estimating epistemic uncertainty by learning to predict generalization error and subtracting an estimate of aleatoric uncertainty, i.e., intrinsic unpredictability. This estimator of epistemic uncertainty includes the effect of model bias and can be applied in non-stationary learning environments arising in active learning or reinforcement learning. In addition to demonstrating these properties of Direct Epistemic Uncertainty Prediction (DEUP), we illustrate its advantage against existing methods for uncertainty estimation on downstream tasks including sequential model optimization and reinforcement learning. We also evaluate the quality of uncertainty estimates from DEUP for probabilistic classification of images and for estimating uncertainty about synergistic drug combinations.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/31/2019

Epistemic Uncertainty Sampling

Various strategies for active learning have been proposed in the machine...
09/02/2021

MACEst: The reliable and trustworthy Model Agnostic Confidence Estimator

Reliable Confidence Estimates are hugely important for any machine learn...
10/05/2022

Query The Agent: Improving sample efficiency through epistemic uncertainty estimation

Curricula for goal-conditioned reinforcement learning agents typically r...
10/25/2022

The uncertainty of infectious disease outbreaks is underestimated

Uncertainty can be classified as either aleatoric (intrinsic randomness)...
07/13/2020

Uncertain-DeepSSM: From Images to Probabilistic Shape Models

Statistical shape modeling (SSM) has recently taken advantage of advance...
01/12/2021

Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion

Following the performance breakthrough of denoising networks, improvemen...
10/18/2021

Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty

Consistent state estimation is challenging, especially under the epistem...