From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

by   Meike Nauta, et al.

The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes, also raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practice of more than 300 papers published in the last 7 years at major AI and ML conferences that introduce an XAI method. We find that 1 in 3 papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate with users. We also contribute to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. This systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark and compare new and existing XAI methods. This also opens up opportunities to include quantitative metrics as optimization criteria during model training in order to optimize for accuracy and interpretability simultaneously.


page 9

page 15

page 18


Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations

The evaluation of explanation methods is a research topic that has not y...

Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches

Explanations in Machine Learning come in many forms, but a consensus reg...

Explaining Any ML Model? – On Goals and Capabilities of XAI

An increasing ubiquity of machine learning (ML) motivates research on al...

The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus

Explainable AI (XAI) is a rapidly evolving field that aims to improve tr...

Explainable Machine Learning using Real, Synthetic and Augmented Fire Tests to Predict Fire Resistance and Spalling of RC Columns

This paper presents the development of systematic machine learning (ML) ...

Explainable AI via Learning to Optimize

Indecipherable black boxes are common in machine learning (ML), but appl...

Please sign up or login with your details

Forgot password? Click here to reset