If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Components

by   Boyue Caroline Hu, et al.

Machine Vision Components (MVC) are becoming safety-critical. Assuring their quality, including safety, is essential for their successful deployment. Assurance relies on the availability of precisely specified and, ideally, machine-verifiable requirements. MVCs with state-of-the-art performance rely on machine learning (ML) and training data but largely lack such requirements. In this paper, we address the need for defining machine-verifiable reliability requirements for MVCs against transformations that simulate the full range of realistic and safety-critical changes in the environment. Using human performance as a baseline, we define reliability requirements as: 'if the changes in an image do not affect a human's decision, neither should they affect the MVC's.' To this end, we provide: (1) a class of safety-related image transformations; (2) reliability requirement classes to specify correctness-preservation and prediction-preservation for MVCs; (3) a method to instantiate machine-verifiable requirements from these requirements classes using human performance experiment data; (4) human performance experiment data for image recognition involving eight commonly used transformations, from about 2000 human participants; and (5) a method for automatically checking whether an MVC satisfies our requirements. Further, we show that our reliability requirements are feasible and reusable by evaluating our methods on 13 state-of-the-art pre-trained image classification models. Finally, we demonstrate that our approach detects reliability gaps in MVCs that other existing methods are unable to detect.


page 2

page 4

page 5

page 10


Justification-Based Reliability in Machine Learning

With the advent of Deep Learning, the field of machine learning (ML) has...

Reliability Assessment and Safety Arguments for Machine Learning Components in Assuring Learning-Enabled Autonomous Systems

The increasing use of Machine Learning (ML) components embedded in auton...

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

High-accurate machine learning (ML) image classifiers cannot guarantee t...

Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262

The use of machine learning (ML) is on the rise in many sectors of softw...

Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems

Due to the increasing usage of machine learning (ML) techniques in secur...

Specifying and Testing k-Safety Properties for Machine-Learning Models

Machine-learning models are becoming increasingly prevalent in our lives...

Two-step counterfactual generation for OOD examples

Two fundamental requirements for the deployment of machine learning mode...

Please sign up or login with your details

Forgot password? Click here to reset