Network Generalization Prediction for Safety Critical Tasks in Novel Operating Domains

by   Molly O'Brien, et al.

It is well known that Neural Network (network) performance often degrades when a network is used in novel operating domains that differ from its training and testing domains. This is a major limitation, as networks are being integrated into safety critical, cyber-physical systems that must work in unconstrained environments, e.g., perception for autonomous vehicles. Training networks that generalize to novel operating domains and that extract robust features is an active area of research, but previous work fails to predict what the network performance will be in novel operating domains. We propose the task Network Generalization Prediction: predicting the expected network performance in novel operating domains. We describe the network performance in terms of an interpretable Context Subspace, and we propose a methodology for selecting the features of the Context Subspace that provide the most information about the network performance. We identify the Context Subspace for a pretrained Faster RCNN network performing pedestrian detection on the Berkeley Deep Drive (BDD) Dataset, and demonstrate Network Generalization Prediction accuracy within 5 or less of observed performance. We also demonstrate that the Context Subspace from the BDD Dataset is informative for completely unseen datasets, JAAD and Cityscapes, where predictions have a bias of 10


Mapping DNN Embedding Manifolds for Network Generalization Prediction

Understanding Deep Neural Network (DNN) performance in changing conditio...

Dependable Neural Networks for Safety Critical Tasks

Neural Networks are being integrated into safety critical systems, e.g.,...

Embedded out-of-distribution detection on an autonomous robot platform

Machine learning (ML) is actively finding its way into modern cyber-phys...

Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Planner

Neural networks have shown great promises in planning, control, and gene...

On Using Blockchains for Safety-Critical Systems

Innovation in the world of today is mainly driven by software. Companies...

Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehicles

Deep learning is the method of choice for trajectory prediction for auto...

Generalizable Classification of UHF Partial Discharge Signals in Gas-Insulated HVDC Systems Using Neural Networks

Undetected partial discharges (PDs) are a safety critical issue in high ...

Please sign up or login with your details

Forgot password? Click here to reset