How to Evaluate Explainability? – A Case for Three Criteria

09/01/2022
by   Timo Speith, et al.
0

The increasing complexity of software systems and the influence of software-supported decisions in our society have sparked the need for software that is safe, reliable, and fair. Explainability has been identified as a means to achieve these qualities. It is recognized as an emerging non-functional requirement (NFR) that has a significant impact on system quality. However, in order to develop explainable systems, we need to understand when a system satisfies this NFR. To this end, appropriate evaluation methods are required. However, the field is crowded with evaluation methods, and there is no consensus on which are the "right" ones. Much less, there is not even agreement on which criteria should be evaluated. In this vision paper, we will provide a multidisciplinary motivation for three such quality criteria concerning the information that systems should provide: comprehensibility, fidelity, and assessability. Our aim is to to fuel the discussion regarding these criteria, such that adequate evaluation methods for them will be conceived.

READ FULL TEXT
research
08/06/2021

Exploring Explainability: A Definition, a Model, and a Knowledge Catalogue

The growing complexity of software systems and the influence of software...
research
07/26/2023

A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)

Within the field of Requirements Engineering (RE), the increasing signif...
research
08/10/2021

Modeling and Evaluating Personas with Software Explainability Requirements

This work focuses on the context of software explainability, which is th...
research
03/25/2022

A Meta Survey of Quality Evaluation Criteria in Explanation Methods

Explanation methods and their evaluation have become a significant issue...
research
12/11/2019

Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches

Explanations in Machine Learning come in many forms, but a consensus reg...
research
02/27/2023

PACCART: Reinforcing Trust in Multiuser Privacy Agreement Systems

Collaborative systems, such as Online Social Networks and the Internet o...
research
01/28/2023

A customizable approach to assess software quality through Multi-Criteria Decision Making

Over the years, Software Quality Engineering has increased interest, dem...

Please sign up or login with your details

Forgot password? Click here to reset