Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

We present a method that "meta" classifies whether segments (objects) predicted by a semantic segmentation neural network intersect with the ground truth. To this end, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene's size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. In our tests, we use two publicly available DeepLabv3+ networks (pre-trained on the Cityscapes data set) and analyze the predictive power of different metrics and different sets of metrics. To this end, we compute logistic LASSO regression fits for the task of classifying IoU=0 vs. IoU > 0 per segment and obtain classification rates of up to 81.91% and AUROC values of up to 87.71% without the incorporation of advanced techniques like Monte-Carlo dropout. We complement these tests with linear regression fits to predict the segment-wise IoU and obtain prediction standard deviations of down to 0.130 as well as R^2 values of up to 81.48%. We show that these results clearly outperform single-metric baseline approaches.

READ FULL TEXT

page 2

page 7

page 9

04/09/2019

Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Images

In the semantic segmentation of street scenes the reliability of the pre...
11/12/2019

Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks

In the semantic segmentation of street scenes, the reliability of a pred...
01/21/2021

Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection

Semantic segmentation aims to robustly predict coherent class labels for...
10/04/2020

MetaDetect: Uncertainty Quantification and Prediction Quality Estimates for Object Detection

In object detection with deep neural networks, the box-wise objectness s...
10/05/2020

MetaBox+: A new Region Based Active Learning Method for Semantic Segmentation using Priority Maps

We present a novel region based active learning method for semantic imag...
12/09/2020

Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation

Deep neural networks (DNNs) for the semantic segmentation of images are ...
05/22/2018

Classification Uncertainty of Deep Neural Networks Based on Gradient Information

We study the quantification of uncertainty of Convolutional Neural Netwo...

Code Repositories