DeepAI AI Chat
Log In Sign Up

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

by   Raul Sena Ferreira, et al.

High-accurate machine learning (ML) image classifiers cannot guarantee that they will not fail at operation. Thus, their deployment in safety-critical applications such as autonomous vehicles is still an open issue. The use of fault tolerance mechanisms such as safety monitors is a promising direction to keep the system in a safe state despite errors of the ML classifier. As the prediction from the ML is the core information directly impacting safety, many works are focusing on monitoring the ML model itself. Checking the efficiency of such monitors in the context of safety-critical applications is thus a significant challenge. Therefore, this paper aims at establishing a baseline framework for benchmarking monitors for ML image classifiers. Furthermore, we propose a framework covering the entire pipeline, from data generation to evaluation. Our approach measures monitor performance with a broader set of metrics than usually proposed in the literature. Moreover, we benchmark three different monitor approaches in 79 benchmark datasets containing five categories of out-of-distribution data for image classifiers: class novelty, noise, anomalies, distributional shifts, and adversarial attacks. Our results indicate that these monitors are no more accurate than a random monitor. We also release the code of all experiments for reproducibility.


page 1

page 2

page 3

page 4


Practical Machine Learning Safety: A Survey and Primer

The open-world deployment of Machine Learning (ML) algorithms in safety-...

SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure

Ensuring safety and explainability of machine learning (ML) is a topic o...

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Context: Machine Learning (ML) has been at the heart of many innovations...

If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Components

Machine Vision Components (MVC) are becoming safety-critical. Assuring t...

Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Driving

Perception is a safety-critical function of autonomous vehicles and mach...

Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search

In Machine Learning (ML)-enabled autonomous systems (MLASs), it is essen...

Adversarial Vulnerability Bounds for Gaussian Process Classification

Machine learning (ML) classification is increasingly used in safety-crit...