Learning to Advise Humans By Leveraging Algorithm Discretion

by   Nicholas Wolczynski, et al.

Expert decision-makers (DMs) in high-stakes AI-advised (AIDeT) settings receive and reconcile recommendations from AI systems before making their final decisions. We identify distinct properties of these settings which are key to developing AIDeT models that effectively benefit team performance. First, DMs in AIDeT settings exhibit algorithm discretion behavior (ADB), i.e., an idiosyncratic tendency to imperfectly accept or reject algorithmic recommendations for any given decision task. Second, DMs incur contradiction costs from exerting decision-making resources (e.g., time and effort) when reconciling AI recommendations that contradict their own judgment. Third, the human's imperfect discretion and reconciliation costs introduce the need for the AI to offer advice selectively. We refer to the task of developing AI to advise humans in AIDeT settings as learning to advise and we address this task by first introducing the AIDeT-Learning Framework. Additionally, we argue that leveraging the human partner's ADB is key to maximizing the AIDeT's decision accuracy while regularizing for contradiction costs. Finally, we instantiate our framework to develop TeamRules (TR): an algorithm that produces rule-based models and recommendations for AIDeT settings. TR is optimized to selectively advise a human and to trade-off contradiction costs and team accuracy for a given environment by leveraging the human partner's ADB. Evaluations on synthetic and real-world benchmark datasets with a variety of simulated human accuracy and discretion behaviors show that TR robustly improves the team's objective across settings over interpretable, rule-based alternatives.


page 1

page 2

page 3

page 4


On the Interdependence of Reliance Behavior and Accuracy in AI-Assisted Decision-Making

In AI-assisted decision-making, a central promise of putting a human in ...

Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

Increasingly, organizations are pairing humans with AI systems to improv...

Optimizing AI for Teamwork

In many high-stakes domains such as criminal justice, finance, and healt...

Value-based Fast and Slow AI Nudging

Nudging is a behavioral strategy aimed at influencing people's thoughts ...

A Q-learning Approach for Adherence-Aware Recommendations

In many real-world scenarios involving high-stakes and safety implicatio...

Learning Complementary Policies for Human-AI Teams

Human-AI complementarity is important when neither the algorithm nor the...

An Interpretable Algorithm for Uveal Melanoma Subtyping from Whole Slide Cytology Images

Algorithmic decision support is rapidly becoming a staple of personalize...

Please sign up or login with your details

Forgot password? Click here to reset