DeepAI AI Chat
Log In Sign Up

The Care Label Concept: A Certification Suite for Trustworthy and Resource-Aware Machine Learning

06/01/2021
by   Katharina Morik, et al.
22

Machine learning applications have become ubiquitous. This has led to an increased effort of making machine learning trustworthy. Explainable and fair AI have already matured. They address knowledgeable users and application engineers. For those who do not want to invest time into understanding the method or the learned model, we offer care labels: easy to understand at a glance, allowing for method or model comparisons, and, at the same time, scientifically well-based. On one hand, this transforms descriptions as given by, e.g., Fact Sheets or Model Cards, into a form that is well-suited for end-users. On the other hand, care labels are the result of a certification suite that tests whether stated guarantees hold. In this paper, we present two experiments with our certification suite. One shows the care labels for configurations of Markov random fields (MRFs). Based on the underlying theory of MRFs, each choice leads to its specific rating of static properties like, e.g., expressivity and reliability. In addition, the implementation is tested and resource consumption is measured yielding dynamic properties. This two-level procedure is followed by another experiment certifying deep neural network (DNN) models. There, we draw the static properties from the literature on a particular model and data set. At the second level, experiments are generated that deliver measurements of robustness against certain attacks. We illustrate this by ResNet-18 and MobileNetV3 applied to ImageNet.

READ FULL TEXT

page 7

page 11

page 13

05/21/2021

Yes We Care! – Certification for Machine Learning Methods through the Care Label Framework

Machine learning applications have become ubiquitous. Their applications...
05/15/2020

Machine Learning as a Catalyst for Value-Based Health Care

In this manuscript, we present an argument that machine learning, a subf...
10/02/2019

Benchmarking machine learning models on eICU critical care dataset

Progress of machine learning in critical care has been difficult to trac...
05/07/2021

Test Suites as a Source of Training Data for Static Analysis Alert Classifiers

Flaw-finding static analysis tools typically generate large volumes of c...
06/06/2017

Adversarial-Playground: A Visualization Suite for Adversarial Sample Generation

With growing interest in adversarial machine learning, it is important f...
08/11/2017

OpenML Benchmarking Suites and the OpenML100

We advocate the use of curated, comprehensive benchmark suites of machin...
11/01/2022

Forecasting Patient Flows with Pandemic Induced Concept Drift using Explainable Machine Learning

Accurately forecasting patient arrivals at Urgent Care Clinics (UCCs) an...