Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

by   Matthias Rottmann, et al.

We present a method that "meta" classifies whether segments (objects) predicted by a semantic segmentation neural network intersect with the ground truth. To this end, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene's size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. In our tests, we use two publicly available DeepLabv3+ networks (pre-trained on the Cityscapes data set) and analyze the predictive power of different metrics and different sets of metrics. To this end, we compute logistic LASSO regression fits for the task of classifying IoU=0 vs. IoU > 0 per segment and obtain classification rates of up to 81.91% and AUROC values of up to 87.71% without the incorporation of advanced techniques like Monte-Carlo dropout. We complement these tests with linear regression fits to predict the segment-wise IoU and obtain prediction standard deviations of down to 0.130 as well as R^2 values of up to 81.48%. We show that these results clearly outperform single-metric baseline approaches.


page 2

page 7

page 9


Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Images

In the semantic segmentation of street scenes the reliability of the pre...

Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks

In the semantic segmentation of street scenes, the reliability of a pred...

False Positive Detection and Prediction Quality Estimation for LiDAR Point Cloud Segmentation

We present a novel post-processing tool for semantic segmentation of LiD...

Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection

Semantic segmentation aims to robustly predict coherent class labels for...

MetaDetect: Uncertainty Quantification and Prediction Quality Estimates for Object Detection

In object detection with deep neural networks, the box-wise objectness s...

Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation

In recent years, deep neural networks have defined the state-of-the-art ...

Classification Uncertainty of Deep Neural Networks Based on Gradient Information

We study the quantification of uncertainty of Convolutional Neural Netwo...

Please sign up or login with your details

Forgot password? Click here to reset