DeepAI AI Chat
Log In Sign Up

Risk Assessment for Machine Learning Models

by   Paul Schwerdtner, et al.

In this paper we propose a framework for assessing the risk associated with deploying a machine learning model in a specified environment. For that we carry over the risk definition from decision theory to machine learning. We develop and implement a method that allows to define deployment scenarios, test the machine learning model under the conditions specified in each scenario, and estimate the damage associated with the output of the machine learning model under test. Using the likelihood of each scenario together with the estimated damage we define key risk indicators of a machine learning model. The definition of scenarios and weighting by their likelihood allows for standardized risk assessment in machine learning throughout multiple domains of application. In particular, in our framework, the robustness of a machine learning model to random input corruptions, distributional shifts caused by a changing environment, and adversarial perturbations can be assessed.


A Modified Drake Equation for Assessing Adversarial Risk to Machine Learning Models

Each machine learning model deployed into production has a risk of adver...

Predicting risk of dyslexia with an online gamified test

Dyslexia is a specific learning disorder related to school failure. Dete...

Holistic risk assessment of inference attacks in machine learning

As machine learning expanding application, there are more and more unign...

Valid Inference for Machine Learning Model Parameters

The parameters of a machine learning model are typically learned by mini...

Selecting Models based on the Risk of Damage Caused by Adversarial Attacks

Regulation, legal liabilities, and societal concerns challenge the adopt...

Position Paper: Towards Transparent Machine Learning

Transparent machine learning is introduced as an alternative form of mac...

Private Machine Learning via Randomised Response

We introduce a general learning framework for private machine learning b...