How to Answer Why – Evaluating the Explanations of AI Through Mental Model Analysis

01/11/2020
by   Tim Schrills, et al.
0

To achieve optimal human-system integration in the context of user-AI interaction it is important that users develop a valid representation of how AI works. In most of the everyday interaction with technical systems users construct mental models (i.e., an abstraction of the anticipated mechanisms a system uses to perform a given task). If no explicit explanations are provided by a system (e.g. by a self-explaining AI) or other sources (e.g. an instructor), the mental model is typically formed based on experiences, i.e. the observations of the user during the interaction. The congruence of this mental model and the actual systems functioning is vital, as it is used for assumptions, predictions and consequently for decisions regarding system use. A key question for human-centered AI research is therefore how to validly survey users' mental models. The objective of the present research is to identify suitable elicitation methods for mental model analysis. We evaluated whether mental models are suitable as an empirical research method. Additionally, methods of cognitive tutoring are integrated. We propose an exemplary method to evaluate explainable AI approaches in a human-centered way.

READ FULL TEXT
research
03/30/2021

Understanding Mental Models of AI through Player-AI Interaction

Designing human-centered AI-driven applications require deep understandi...
research
02/18/2022

A Mental-Model Centric Landscape of Human-AI Symbiosis

There has been significant recent interest in developing AI agents capab...
research
05/23/2023

What Else Do I Need to Know? The Effect of Background Information on Users' Reliance on AI Systems

AI systems have shown impressive performance at answering questions by r...
research
02/20/2022

Direct assessment of individual connotation and experience: An introduction to cognitive-affective mapping

We introduce cognitive-affective maps (CAMs) as a novel tool to assess i...
research
03/10/2023

Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbook

Deployed artificial intelligence (AI) often impacts humans, and there is...
research
02/22/2022

Towards User-Centered Metrics for Trustworthy AI in Immersive Cyberspace

AI plays a key role in current cyberspace and future immersive ecosystem...
research
12/21/2022

Classifying Mental-Disorders through Clinicians Subjective Approach based on Three-way Decision

In psychiatric diagnosis, a contemporary data-driven, manual-based metho...

Please sign up or login with your details

Forgot password? Click here to reset