Robust Neural Regression via Uncertainty Learning

10/12/2021
by   Akib Mashrur, et al.
0

Deep neural networks tend to underestimate uncertainty and produce overly confident predictions. Recently proposed solutions, such as MC Dropout and SDENet, require complex training and/or auxiliary out-of-distribution data. We propose a simple solution by extending the time-tested iterative reweighted least square (IRLS) in generalised linear regression. We use two sub-networks to parametrise the prediction and uncertainty estimation, enabling easy handling of complex inputs and nonlinear response. The two sub-networks have shared representations and are trained via two complementary loss functions for the prediction and the uncertainty estimates, with interleaving steps as in a cooperative game. Compared with more complex models such as MC-Dropout or SDE-Net, our proposed network is simpler to implement and more robust (insensitive to varying aleatoric and epistemic uncertainty).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/03/2021

A Bayesian Neural Network based on Dropout Regulation

Bayesian Neural Networks (BNN) have recently emerged in the Deep Learnin...
research
10/07/2021

Improving MC-Dropout Uncertainty Estimates with Calibration Error-based Optimization

Uncertainty quantification of machine learning and deep learning methods...
research
07/07/2020

Single Shot MC Dropout Approximation

Deep neural networks (DNNs) are known for their high prediction performa...
research
09/13/2018

Deep Network Uncertainty Maps for Indoor Navigation

Estimating the uncertainty of predictions is a crucial ability for robot...
research
07/15/2021

Randomized ReLU Activation for Uncertainty Estimation of Deep Neural Networks

Deep neural networks (DNNs) have successfully learned useful data repres...
research
10/07/2019

Evaluating Scalable Uncertainty Estimation Methods for DNN-Based Molecular Property Prediction

Advances in deep neural network (DNN) based molecular property predictio...
research
12/15/2020

Masksembles for Uncertainty Estimation

Deep neural networks have amply demonstrated their prowess but estimatin...

Please sign up or login with your details

Forgot password? Click here to reset