A Hierarchical HAZOP-Like Safety Analysis for Learning-Enabled Systems

by   Yi Qi, et al.

Hazard and Operability Analysis (HAZOP) is a powerful safety analysis technique with a long history in industrial process control domain. With the increasing use of Machine Learning (ML) components in cyber physical systems–so called Learning-Enabled Systems (LESs), there is a recent trend of applying HAZOP-like analysis to LESs. While it shows a great potential to reserve the capability of doing sufficient and systematic safety analysis, there are new technical challenges raised by the novel characteristics of ML that require retrofit of the conventional HAZOP technique. In this regard, we present a new Hierarchical HAZOP-Like method for LESs (HILLS). To deal with the complexity of LESs, HILLS first does "divide and conquer" by stratifying the whole system into three levels, and then proceeds HAZOP on each level to identify (latent-)hazards, causes, security threats and mitigation (with new nodes and guide words). Finally, HILLS attempts at linking and propagating the causal relationship among those identified elements within and across the three levels via both qualitative and quantitative methods. We examine and illustrate the utility of HILLS by a case study on Autonomous Underwater Vehicles, with discussions on assumptions and extensions to real-world applications. HILLS, as a first HAZOP-like attempt on LESs that explicitly considers ML internal behaviours and its interactions with other components, not only uncovers the inherent difficulties of doing safety analysis for LESs, but also demonstrates a good potential to tackle them.


Towards Developing Safety Assurance Cases for Learning-Enabled Medical Cyber-Physical Systems

Machine Learning (ML) technologies have been increasingly adopted in Med...

Reliability Assessment and Safety Arguments for Machine Learning Components in Assuring Learning-Enabled Autonomous Systems

The increasing use of Machine Learning (ML) components embedded in auton...

MDE for Machine Learning-Enabled Software Systems: A Case Study and Comparison of MontiAnna ML-Quadrat

In this paper, we propose to adopt the MDE paradigm for the development ...

Quantifying Assurance in Learning-enabled Systems

Dependability assurance of systems embedding machine learning(ML) compon...

The Role of Machine Learning in Cybersecurity

Machine Learning (ML) represents a pivotal technology for current and fu...

Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search

In Machine Learning (ML)-enabled autonomous systems (MLASs), it is essen...

A Framework for Assurance of Medication Safety using Machine Learning

Medication errors continue to be the leading cause of avoidable patient ...

Please sign up or login with your details

Forgot password? Click here to reset