Inhibited Softmax for Uncertainty Estimation in Neural Networks

10/03/2018
by   Marcin Możejko, et al.
7

We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains.

READ FULL TEXT

page 8

page 13

research
07/04/2022

Variational Neural Networks

Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertain...
research
10/25/2022

Revisiting Softmax for Uncertainty Approximation in Text Classification

Uncertainty approximation in text classification is an important area wi...
research
05/27/2019

Radial Prediction Layer

For a broad variety of critical applications, it is essential to know ho...
research
02/11/2020

Fine-grained Uncertainty Modeling in Neural Networks

Existing uncertainty modeling approaches try to detect an out-of-distrib...
research
11/21/2021

Efficient Softmax Approximation for Deep Neural Networks with Attention Mechanism

There has been a rapid advance of custom hardware (HW) for accelerating ...
research
05/19/2017

Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML

Reward augmented maximum likelihood (RAML), a simple and effective learn...

Please sign up or login with your details

Forgot password? Click here to reset