DeepAI AI Chat
Log In Sign Up

Theory of Mind for Deep Reinforcement Learning in Hanabi

by   Andrew Fuchs, et al.

The partially observable card game Hanabi has recently been proposed as a new AI challenge problem due to its dependence on implicit communication conventions and apparent necessity of theory of mind reasoning for efficient play. In this work, we propose a mechanism for imbuing Reinforcement Learning agents with a theory of mind to discover efficient cooperative strategies in Hanabi. The primary contributions of this work are threefold: First, a formal definition of a computationally tractable mechanism for computing hand probabilities in Hanabi. Second, an extension to conventional Deep Reinforcement Learning that introduces reasoning over finitely nested theory of mind belief hierarchies. Finally, an intrinsic reward mechanism enabled by theory of mind that incentivizes agents to share strategically relevant private knowledge with their teammates. We demonstrate the utility of our algorithm against Rainbow, a state-of-the-art Reinforcement Learning agent.


page 1

page 2

page 3

page 4


Theory of Mind with Guilt Aversion Facilitates Cooperative Reinforcement Learning

Guilt aversion induces experience of a utility loss in people if they be...

Efficient Reinforcement Learning with a Mind-Game for Full-Length StarCraft II

StarCraft II provides an extremely challenging platform for reinforcemen...

Modeling Theory of Mind for Autonomous Agents with Probabilistic Programs

As autonomous agents become more ubiquitous, they will eventually have t...

Unbiased Self-Play

We present a general optimization framework for emergent belief-state re...

A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity

The key challenge in multiagent learning is learning a best response to ...

A Definition of Happiness for Reinforcement Learning Agents

What is happiness for reinforcement learning agents? We seek a formal de...

Learning Implicit Communication Strategies for the Purpose of Illicit Collusion

Winner-take-all dynamics are prevalent throughout the human and natural ...