On the Utility of Learning about Humans for Human-AI Coordination

10/13/2019
by   Micah Carroll, et al.
8

While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves. Agents that assume their partner to be optimal or similar to them can converge to coordination protocols that fail to understand and be understood by humans. To demonstrate this, we introduce a simple environment that requires challenging coordination, based on the popular game Overcooked, and learn a simple model that mimics human play. We evaluate the performance of agents trained via self-play and population-based training. These agents perform very well when paired with themselves, but when paired with our human model, they are significantly worse than agents designed to play with the human model. An experiment with a planning algorithm yields the same conclusion, though only when the human-aware planner is given the exact human model that it is playing with. A user study with real humans shows this pattern as well, though less strongly. Qualitatively, we find that the gains come from having the agent adapt to the human's gameplay. Given this result, we suggest several approaches for designing agents that learn about humans in order to better coordinate with them. Code is available at https://github.com/HumanCompatibleAI/overcooked_ai.

READ FULL TEXT

page 4

page 8

page 18

research
10/15/2021

Collaborating with Humans without Human Data

Collaborating with humans requires rapidly adapting to their individual ...
research
01/16/2023

PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination

Zero-shot human-AI coordination holds the promise of collaborating with ...
research
05/02/2018

AI safety via debate

To make AI systems broadly useful for challenging real-world tasks, we n...
research
02/11/2016

Modeling Human Ad Hoc Coordination

Whether in groups of humans or groups of computer agents, collaboration ...
research
01/29/2022

Learning to Coordinate with Humans using Action Features

An unaddressed challenge in human-AI coordination is to enable AI agents...
research
04/04/2022

A Machine With Human-Like Memory Systems

Inspired by the cognitive science theory, we explicitly model an agent w...
research
04/07/2021

On the Critical Role of Conventions in Adaptive Human-AI Collaboration

Humans can quickly adapt to new partners in collaborative tasks (e.g. pl...

Please sign up or login with your details

Forgot password? Click here to reset