Engineering Safety in Machine Learning

01/16/2016
by   Kush R. Varshney, et al.
0

Machine learning algorithms are increasingly influencing our decisions and interacting with us in all parts of our daily lives. Therefore, just like for power plants, highways, and myriad other engineered sociotechnical systems, we must consider the safety of systems involving machine learning. In this paper, we first discuss the definition of safety in terms of risk, epistemic uncertainty, and the harm incurred by unwanted outcomes. Then we examine dimensions, such as the choice of cost function and the appropriateness of minimizing the empirical average training cost, along which certain real-world applications may not be completely amenable to the foundational principle of modern statistical machine learning: empirical risk minimization. In particular, we note an emerging dichotomy of applications: ones in which safety is important and risk minimization is not the complete story (we name these Type A applications), and ones in which safety is not so critical and risk minimization is sufficient (we name these Type B applications). Finally, we discuss how four different strategies for achieving safety in engineering (inherently safe design, safety reserves, safe fail, and procedural safeguards) can be mapped to the machine learning context through interpretability and causality of predictive models, objectives beyond expected prediction accuracy, human involvement for labeling difficult or rare examples, and user experience design of software.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/05/2016

On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products

Machine learning algorithms increasingly influence our decisions and int...
research
05/24/2023

Taylor Learning

Empirical risk minimization stands behind most optimization in supervise...
research
02/04/2022

A Survey on Safety-Critical Driving Scenario Generation – A Methodological Perspective

Autonomous driving systems have witnessed a significant development duri...
research
08/04/2020

Safety design concepts for statistical machine learning components toward accordance with functional safety standards

In recent years, curial incidents and accidents have been reported due t...
research
08/02/2018

The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamic Systems

Learning algorithms have shown considerable prowess in simulation by all...
research
07/11/2022

From Correlation to Causation: Formalizing Interpretable Machine Learning as a Statistical Process

Explainable AI (XAI) is a necessity in safety-critical systems such as i...
research
02/21/2023

Reentry Risk and Safety Assessment of Spacecraft Debris Based on Machine Learning

Uncontrolled spacecraft will disintegrate and generate a large amount of...

Please sign up or login with your details

Forgot password? Click here to reset