DeepAI AI Chat
Log In Sign Up

On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection

by   Vivian Lai, et al.
University of Colorado Boulder

Humans are the final decision makers in critical tasks that involve ethical and legal concerns, ranging from recidivism prediction, to medical diagnosis, to fighting against fake news. Although machine learning models can sometimes achieve impressive performance in these tasks, these tasks are not amenable to full automation. To realize the potential of machine learning for improving human decisions, it is important to understand how assistance from machine learning models affect human performance and human agency. In this paper, we use deception detection as a testbed and investigate how we can harness explanations and predictions of machine learning models to improve human performance while retaining human agency. We propose a spectrum between full human agency and full automation, and develop varying levels of machine assistance along the spectrum that gradually increase the influence of machine predictions. We find that without showing predicted labels, explanations alone do not statistically significantly improve human performance in the end task. In comparison, human performance is greatly improved by showing predicted labels (>20 suggesting strong machine performance. Interestingly, when predicted labels are shown, explanations of machine predictions induce a similar level of accuracy as an explicit statement of strong machine performance. Our results demonstrate a tradeoff between human performance and human agency and show that explanations of machine predictions can moderate this tradeoff.


page 5

page 6

page 7

page 8

page 11

page 12


Harnessing Explanations to Bridge AI and Humans

Machine learning models are increasingly integrated into societally crit...

What to Learn, and How: Toward Effective Learning from Rationales

Learning from rationales seeks to augment model training with human-prov...

Symbolic Knowledge Extraction from Opaque Predictors Applied to Cosmic-Ray Data Gathered with LISA Pathfinder

Machine learning models are nowadays ubiquitous in space missions, perfo...

Regression Under Human Assistance

Decisions are increasingly taken by both humans and machine learning mod...

Margin-distancing for safe model explanation

The growing use of machine learning models in consequential settings has...

"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans

To support human decision making with machine learning models, we often ...

Assisting Human Decisions in Document Matching

Many practical applications, ranging from paper-reviewer assignment in p...