Learning Generalizable Risk-Sensitive Policies to Coordinate in Decentralized Multi-Agent General-Sum Games

05/31/2022
by   Ziyi Liu, et al.
0

While various multi-agent reinforcement learning methods have been proposed in cooperative settings, few works investigate how self-interested learning agents achieve mutual coordination in decentralized general-sum games and generalize pre-trained policies to non-cooperative opponents during execution. In this paper, we present a generalizable and sample efficient algorithm for multi-agent coordination in decentralized general-sum games without any access to other agents' rewards or observations. Specifically, we first learn the distributions over the return of individuals and estimate a dynamic risk-seeking bonus to encourage agents to discover risky coordination strategies. Furthermore, to avoid overfitting opponents' coordination strategies during training, we propose an auxiliary opponent modeling task so that agents can infer their opponents' type and dynamically alter corresponding strategies during execution. Empirically, we show that agents trained via our method can achieve mutual coordination during training and avoid being exploited by non-cooperative opponents during execution, which outperforms other baseline methods and reaches the state-of-the-art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/14/2023

Learning Adaptable Risk-Sensitive Policies to Coordinate in Multi-Agent General-Sum Games

In general-sum games, the interaction of self-interested learning agents...
research
02/16/2021

RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agents

Current value-based multi-agent reinforcement learning methods optimize ...
research
03/15/2023

Coordinating Fully-Cooperative Agents Using Hierarchical Learning Anticipation

Learning anticipation is a reasoning paradigm in multi-agent reinforceme...
research
04/05/2023

Emergent Coordination through Game-Induced Nonlinear Opinion Dynamics

We present a multi-agent decision-making framework for the emergent coor...
research
04/18/2013

Interactive POMDP Lite: Towards Practical Planning to Predict and Exploit Intentions for Interacting with Self-Interested Agents

A key challenge in non-cooperative multi-agent systems is that of develo...
research
08/18/2023

Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models

Developing effective Multi-Agent Systems (MAS) is critical for many appl...
research
04/10/2022

MA-Dreamer: Coordination and communication through shared imagination

Multi-agent RL is rendered difficult due to the non-stationary nature of...

Please sign up or login with your details

Forgot password? Click here to reset