System Safety Engineering for Social and Ethical ML Risks: A Case Study

11/08/2022
by   Edgar W. Jatho III, et al.
0

Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems, with a particular focus on social and ethical risks of ML components in complex sociotechnical systems. However, existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. Systems safety engineering is a well established discipline with a track record of identifying and managing risks in many complex sociotechnical domains. We adopt the natural hypothesis that tools from this domain could serve to enhance risk analyses of ML in its context of use. To test this hypothesis, we apply a "best of breed" systems safety analysis, Systems Theoretic Process Analysis (STPA), to a specific high-consequence system with an important ML-driven component, namely the Prescription Drug Monitoring Programs (PDMPs) operated by many US States, several of which rely on an ML-derived risk score. We focus in particular on how this analysis can extend to identifying social and ethical risks and developing concrete design-level controls to mitigate them.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2023

Concrete Safety for ML Problems: System Safety for ML Development and Assessment

Many stakeholders struggle to make reliances on ML-driven systems due to...
research
07/19/2023

Beyond the ML Model: Applying Safety Engineering Frameworks to Text-to-Image Development

Identifying potential social and ethical risks in emerging machine learn...
research
10/30/2020

Waymo's Safety Methodologies and Safety Readiness Determinations

Waymo's safety methodologies, which draw on well established engineering...
research
10/06/2022

From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML

Inappropriate design and deployment of machine learning (ML) systems lea...
research
04/21/2023

The Dark Side of ChatGPT: Legal and Ethical Challenges from Stochastic Parrots and Hallucination

With the launch of ChatGPT, Large Language Models (LLMs) are shaking up ...
research
01/30/2023

Exploring AI Ethics of ChatGPT: A Diagnostic Analysis

Recent breakthroughs in natural language processing (NLP) have permitted...
research
05/02/2022

Modeling and mitigation of occupational safety risks in dynamic industrial environments

Identifying and mitigating safety risks is paramount in a number of indu...

Please sign up or login with your details

Forgot password? Click here to reset