Unbiased Self-Play

06/06/2021

∙

We present a general optimization framework for emergent belief-state representation without any supervision. We employed the common configuration of multiagent reinforcement learning and communication to improve exploration coverage over an environment by leveraging the knowledge of each agent. In this paper, we obtained that recurrent neural nets (RNNs) with shared weights are highly biased in partially observable environments because of their noncooperativity. To address this, we designated an unbiased version of self-play via mechanism design, also known as reverse game theory, to clarify unbiased knowledge at the Bayesian Nash equilibrium. The key idea is to add imaginary rewards using the peer prediction mechanism, i.e., a mechanism for mutually criticizing information in a decentralized environment. Numerical analyses, including StarCraft exploration tasks with up to 20 agents and off-the-shelf RNNs, demonstrate the state-of-the-art performance.

READ FULL TEXT

Unbiased Self-Play

Memory Augmented Self-Play

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Theory of Mind for Deep Reinforcement Learning in Hanabi

Linear Convergent Distributed Nash Equilibrium Seeking with Compression

Implicit Posterior Variational Inference for Deep Gaussian Processes

Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

Recurrent Control Nets for Deep Reinforcement Learning

Unbiased Self-Play

Related Research

Memory Augmented Self-Play

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Theory of Mind for Deep Reinforcement Learning in Hanabi

Linear Convergent Distributed Nash Equilibrium Seeking with Compression

Implicit Posterior Variational Inference for Deep Gaussian Processes

Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

Recurrent Control Nets for Deep Reinforcement Learning