DeepAI AI Chat
Log In Sign Up

DEUP: Direct Epistemic Uncertainty Prediction

by   Moksh Jain, et al.

Epistemic uncertainty is the part of out-of-sample prediction error due to the lack of knowledge of the learner. Whereas previous work was focusing on model variance, we propose a principled approach for directly estimating epistemic uncertainty by learning to predict generalization error and subtracting an estimate of aleatoric uncertainty, i.e., intrinsic unpredictability. This estimator of epistemic uncertainty includes the effect of model bias and can be applied in non-stationary learning environments arising in active learning or reinforcement learning. In addition to demonstrating these properties of Direct Epistemic Uncertainty Prediction (DEUP), we illustrate its advantage against existing methods for uncertainty estimation on downstream tasks including sequential model optimization and reinforcement learning. We also evaluate the quality of uncertainty estimates from DEUP for probabilistic classification of images and for estimating uncertainty about synergistic drug combinations.


page 1

page 2

page 3

page 4


Epistemic Uncertainty Sampling

Various strategies for active learning have been proposed in the machine...

MACEst: The reliable and trustworthy Model Agnostic Confidence Estimator

Reliable Confidence Estimates are hugely important for any machine learn...

Query The Agent: Improving sample efficiency through epistemic uncertainty estimation

Curricula for goal-conditioned reinforcement learning agents typically r...

The uncertainty of infectious disease outbreaks is underestimated

Uncertainty can be classified as either aleatoric (intrinsic randomness)...

Uncertain-DeepSSM: From Images to Probabilistic Shape Models

Statistical shape modeling (SSM) has recently taken advantage of advance...

Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion

Following the performance breakthrough of denoising networks, improvemen...

Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty

Consistent state estimation is challenging, especially under the epistem...