Multi-Agent Generalized Recursive Reasoning

01/26/2019
by   Ying Wen, et al.
16

We propose a new reasoning protocol called generalized recursive reasoning (GR2), and embed it into the multi-agent reinforcement learning (MARL) framework. The GR2 model defines reasoning categories: level-0 agent acts randomly, and level-k agent takes the best response to a mixed type of agents that are distributed over level 0 to k-1. The GR2 leaners can take into account the bounded rationality, and it does not need the assumption that the opponent agents play Nash strategy in all stage games, which many MARL algorithms require. We prove that when the level k is large, the GR2 learners will converge to at least one Nash Equilibrium (NE). In addition, if lower-level agents play the NE, high-level agents will surely follow as well. We evaluate the GR2 Soft Actor-Critic algorithms in a series of games and high-dimensional environment; results show that the GR2 methods have faster convergence speed than strong MARL baselines.

READ FULL TEXT
research
09/08/2019

Bi-level Actor-Critic for Multi-agent Coordination

Coordination is one of the essential problems in multi-agent systems. Ty...
research
01/26/2019

Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning

Humans are capable of attributing latent mental contents such as beliefs...
research
06/30/2020

R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games

This paper presents a recursive reasoning formalism of Bayesian optimiza...
research
09/02/2021

Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts

Multi-agent inverse reinforcement learning (MIRL) can be used to learn r...
research
06/13/2012

Identifying reasoning patterns in games

We present an algorithm that identifies the reasoning patterns of agents...
research
09/13/2023

Distributed Learning Dynamics for Coalitional Games

In the framework of transferable utility coalitional games, a scoring (c...
research
08/04/2017

Routing Games over Time with FIFO policy

We study atomic routing games where every agent travels both along its d...

Please sign up or login with your details

Forgot password? Click here to reset