Towards Risk Modeling for Collaborative AI

by   Matteo Camilli, et al.

Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements domain specific standards and regulations is of greatest importance. Challenges associated with the achievement of this goal become even more severe when such systems rely on machine learning components rather than such as top-down rule-based AI. In this paper, we introduce a risk modeling approach tailored to Collaborative AI systems. The risk model includes goals, risk events and domain specific indicators that potentially expose humans to hazards. The risk model is then leveraged to drive assurance methods that feed in turn the risk model through insights extracted from run-time evidence. Our envisioned approach is described by means of a running example in the domain of Industry 4.0, where a robotic arm endowed with a visual perception component, implemented with machine learning, collaborates with a human operator for a production-relevant task.



There are no comments yet.


page 1

page 2

page 3

page 4


Collaborative AI Needs Stronger Assurances Driven by Risks

Collaborative AI systems (CAISs) aim at working together with humans in ...

Excavating "Excavating AI": The Elephant in the Gallery

Contains critical commentary on the exhibitions "Training Humans" and "M...

Teaming up with information agents

Despite the intricacies involved in designing a computer as a teampartne...

Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi

Deep reinforcement learning has generated superhuman AI in competitive g...

Hybrid Approach to Automation, RPA and Machine Learning: a Method for the Human-centered Design of Software Robots

One of the more prominent trends within Industry 4.0 is the drive to emp...

Screenplay Quality Assessment: Can We Predict Who Gets Nominated?

Deciding which scripts to turn into movies is a costly and time-consumin...

Look Before You Leap! Designing a Human-Centered AI System for Change Risk Assessment

Reducing the number of failures in a production system is one of the mos...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.