Classification Uncertainty of Deep Neural Networks Based on Gradient Information

05/22/2018
by   Philipp Oberdiek, et al.
0

We study the quantification of uncertainty of Convolutional Neural Networks (CNNs) based on gradient metrics. Unlike the classical softmax entropy, such metrics gather information from all layers of the CNN. We show for the (E)MNIST data set that for several such metrics we achieve the same meta classification accuracy -- i.e. the task of classifying correctly predicted labels as correct and incorrectly predicted ones as incorrect without knowing the actual label -- as for entropy thresholding. Meta classification rates for out of sample images can be increased when using entropy together with several gradient based metrics as input quantities for a meta-classifier. This proves that our gradient based metrics do not contain the same information as the entropy. We also apply meta classification to concepts not used during training: EMNIST/Omniglot letters, CIFAR10 and noise. Meta classifiers only trained on the uncertainty metrics of classes available during training usually do not perform equally well for all the unknown concepts letters, CIFAR10 and uniform noise. If we however allow the meta classifier to be trained on uncertainty metrics including some samples of some or all of the categories, meta classification for concepts remote from MNIST digits can be improved considerably.

READ FULL TEXT
research
07/03/2019

Measuring the Data Efficiency of Deep Learning Methods

In this paper, we propose a new experimental protocol and use it to benc...
research
06/10/2019

Quantifying Layerwise Information Discarding of Neural Networks

This paper presents a method to explain how input information is discard...
research
09/17/2022

A study on the deviations in performance of FNNs and CNNs in the realm of grayscale adversarial images

Neural Networks are prone to having lesser accuracy in the classificatio...
research
04/24/2020

DeepMerge: Classifying High-redshift Merging Galaxies with Deep Neural Networks

We investigate and demonstrate the use of convolutional neural networks ...
research
04/09/2019

Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Images

In the semantic segmentation of street scenes the reliability of the pre...
research
11/01/2018

Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

We present a method that "meta" classifies whether segments (objects) pr...
research
11/12/2019

Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks

In the semantic segmentation of street scenes, the reliability of a pred...

Please sign up or login with your details

Forgot password? Click here to reset