Learning Complementary Policies for Human-AI Teams

02/06/2023
by   Ruijiang Gao, et al.
0

Human-AI complementarity is important when neither the algorithm nor the human yields dominant performance across all instances in a given context. Recent work that explored human-AI collaboration has considered decisions that correspond to classification tasks. However, in many important contexts where humans can benefit from AI complementarity, humans undertake course of action. In this paper, we propose a framework for a novel human-AI collaboration for selecting advantageous course of action, which we refer to as Learning Complementary Policy for Human-AI teams (lcp-hai). Our solution aims to exploit the human-AI complementarity to maximize decision rewards by learning both an algorithmic policy that aims to complement humans by a routing model that defers decisions to either a human or the AI to leverage the resulting complementarity. We then extend our approach to leverage opportunities and mitigate risks that arise in important contexts in practice: 1) when a team is composed of multiple humans with differential and potentially complementary abilities, 2) when the observational data includes consistent deterministic actions, and 3) when the covariate distribution of future decisions differ from that in the historical data. We demonstrate the effectiveness of our proposed methods using data on real human responses and semi-synthetic, and find that our methods offer reliable and advantageous performance across setting, and that it is superior to when either the algorithm or the AI make decisions on their own. We also find that the extensions we propose effectively improve the robustness of the human-AI collaboration performance in the presence of different challenging settings.

READ FULL TEXT
research
05/22/2021

Human-AI Collaboration with Bandit Feedback

Human-machine complementarity is important when neither the algorithm no...
research
04/29/2022

Doubting AI Predictions: Influence-Driven Second Opinion Recommendation

Effective human-AI collaboration requires a system design that provides ...
research
03/16/2023

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

Recent work has proposed artificial intelligence (AI) models that can le...
research
07/28/2022

Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables

In many real world contexts, successful human-AI collaboration requires ...
research
10/02/2021

Partner-Aware Algorithms in Decentralized Cooperative Bandit Teams

When humans collaborate with each other, they often make decisions by ob...
research
10/23/2022

Learning to Advise Humans By Leveraging Algorithm Discretion

Expert decision-makers (DMs) in high-stakes AI-advised (AIDeT) settings ...
research
10/28/2019

Human-AI Co-Learning for Data-Driven AI

Human and AI are increasingly interacting and collaborating to accomplis...

Please sign up or login with your details

Forgot password? Click here to reset