Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

We present a method that "meta" classifies whether segments (objects) predicted by a semantic segmentation neural network intersect with the ground truth. To this end, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene's size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. In our tests, we use two publicly available DeepLabv3+ networks (pre-trained on the Cityscapes data set) and analyze the predictive power of different metrics and different sets of metrics. To this end, we compute logistic LASSO regression fits for the task of classifying IoU=0 vs. IoU > 0 per segment and obtain classification rates of up to 81.91% and AUROC values of up to 87.71% without the incorporation of advanced techniques like Monte-Carlo dropout. We complement these tests with linear regression fits to predict the segment-wise IoU and obtain prediction standard deviations of down to 0.130 as well as R^2 values of up to 81.48%. We show that these results clearly outperform single-metric baseline approaches.


page 2

page 7

page 9


Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Images

In the semantic segmentation of street scenes the reliability of the pre...

Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks

In the semantic segmentation of street scenes, the reliability of a pred...

Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection

Semantic segmentation aims to robustly predict coherent class labels for...

MetaDetect: Uncertainty Quantification and Prediction Quality Estimates for Object Detection

In object detection with deep neural networks, the box-wise objectness s...

MetaBox+: A new Region Based Active Learning Method for Semantic Segmentation using Priority Maps

We present a novel region based active learning method for semantic imag...

Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation

Deep neural networks (DNNs) for the semantic segmentation of images are ...

Classification Uncertainty of Deep Neural Networks Based on Gradient Information

We study the quantification of uncertainty of Convolutional Neural Netwo...

Code Repositories