DeepAI AI Chat
Log In Sign Up

"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans

01/14/2020
by   Vivian Lai, et al.
University of Colorado Boulder
Association for Computing Machinery
8

To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans. While existing approaches focus on explaining machine predictions with real-time assistance, we explore model-driven tutorials to help humans understand these patterns in a training phase. We consider both tutorials with guidelines from scientific papers, analogous to current practices of science communication, and automatically selected examples from training data with explanations. We use deceptive review detection as a testbed and conduct large-scale, randomized human-subject experiments to examine the effectiveness of such tutorials. We find that tutorials indeed improve human performance, with and without real-time assistance. In particular, although deep learning provides superior predictive performance than simple models, tutorials and explanations from simple models are more useful to humans. Our work suggests future directions for human-centered tutorials and explanations towards a synergy between humans and AI.

READ FULL TEXT

page 7

page 8

page 9

page 16

page 17

page 18

page 19

page 20

03/16/2020

Harnessing Explanations to Bridge AI and Humans

Machine learning models are increasingly integrated into societally crit...
10/13/2022

Self-explaining deep models with logic rule reasoning

We present SELOR, a framework for integrating self-explaining capabiliti...
01/13/2021

Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making

Although AI holds promise for improving human decision making in societa...
04/26/2022

Process Knowledge-infused Learning for Suicidality Assessment on Social Media

Improving the performance and natural language explanations of deep lear...
02/16/2022

SoK: Human-Centered Phishing Susceptibility

Phishing is recognised as a serious threat to organisations and individu...
09/06/2019

Regression Under Human Assistance

Decisions are increasingly taken by both humans and machine learning mod...