Log In Sign Up

Improving Online Performance Prediction for Semantic Segmentation

by   Marvin Klingner, et al.

In this work we address the task of observing the performance of a semantic segmentation deep neural network (DNN) during online operation, i.e., during inference, which is of high importance in safety-critical applications such as autonomous driving. Here, many high-level decisions rely on such DNNs, which are usually evaluated offline, while their performance in online operation remains unknown. To solve this problem, we propose an improved online performance prediction scheme, building on a recently proposed concept of predicting the primary semantic segmentation task's performance. This can be achieved by evaluating the auxiliary task of monocular depth estimation with a measurement supplied by a LiDAR sensor and a subsequent regression to the semantic segmentation performance. In particular, we propose (i) sequential training methods for both tasks in a multi-task training setup, (ii) to share the encoder as well as parts of the decoder between both task's networks for improved efficiency, and (iii) a temporal statistics aggregation method, which significantly reduces the performance prediction error at the cost of a small algorithmic latency. Evaluation on the KITTI dataset shows that all three aspects improve the performance prediction compared to previous approaches.


page 1

page 3

page 5


Exploiting Test Time Evidence to Improve Predictions of Deep Neural Networks

Many prediction tasks, especially in computer vision, are often inherent...

AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving

Decision making in automated driving is highly specific to the environme...

Are We Hungry for 3D LiDAR Data for Semantic Segmentation?

3D LiDAR semantic segmentation is a pivotal task that is widely involved...

Generalizing Interactive Backpropagating Refinement for Dense Prediction

As deep neural networks become the state-of-the-art approach in the fiel...

Learning a Domain-Agnostic Visual Representation for Autonomous Driving via Contrastive Loss

Deep neural networks have been widely studied in autonomous driving appl...

Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving

Deep Neural Networks (DNNs) are rapidly being adopted by the automotive ...