Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals

by   Weina Jin, et al.

Non-technical end-users are silent and invisible users of the state-of-the-art explainable artificial intelligence (XAI) technologies. Their demands and requirements for AI explainability are not incorporated into the design and evaluation of XAI techniques, which are developed to explain the rationales of AI decisions to end-users and assist their critical decisions. This makes XAI techniques ineffective or even harmful in high-stakes applications, such as healthcare, criminal justice, finance, and autonomous driving systems. To systematically understand end-users' requirements to support the technical development of XAI, we conducted the EUCA user study with 32 layperson participants in four AI-assisted critical tasks. The study identified comprehensive user requirements for feature-, example-, and rule-based XAI techniques (manifested by the end-user-friendly explanation forms) and XAI evaluation objectives (manifested by the explanation goals), which were shown to be helpful to directly inspire the proposal of new XAI algorithms and evaluation metrics. The EUCA study findings, the identified explanation forms and goals for technical specification, and the EUCA study dataset support the design and evaluation of end-user-centered XAI techniques for accessible, safe, and accountable AI.


page 6

page 7


Transcending XAI Algorithm Boundaries through End-User-Inspired Design

The boundaries of existing explainable artificial intelligence (XAI) alg...

EUCA: A Practical Prototyping Framework towards End-User-Centered Explainable Artificial Intelligence

The ability to explain decisions to its end-users is a necessity to depl...

Principles of Explanation in Human-AI Systems

Explainable Artificial Intelligence (XAI) has re-emerged in response to ...

What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps

In healthcare, AI systems support clinicians and patients in diagnosis, ...

Question-Driven Design Process for Explainable AI User Experiences

A pervasive design issue of AI systems is their explainability–how to pr...

Toward Foraging for Understanding of StarCraft Agents: An Empirical Study

Assessing and understanding intelligent agents is a difficult task for u...

Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML

Automated Machine Learning (AutoML) is a rapidly growing set of technolo...

Please sign up or login with your details

Forgot password? Click here to reset