PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration

by   Pengyi Li, et al.

Learning to collaborate is critical in Multi-Agent Reinforcement Learning (MARL). Previous works promote collaboration by maximizing the correlation of agents' behaviors, which is typically characterized by Mutual Information (MI) in different forms. However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. PMIC uses a new collaboration criterion measured by the MI between global states and joint actions. Based on this criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating better collaborations while avoiding falling into sub-optimal ones. Experiments on a wide range of MARL benchmarks show the superior performance of PMIC compared with other algorithms.



page 7


A Maximum Mutual Information Framework for Multi-Agent Reinforcement Learning

In this paper, we propose a maximum mutual information (MMI) framework f...

Measuring collaborative emergent behavior in multi-agent reinforcement learning

Multi-agent reinforcement learning (RL) has important implications for t...

Maximum Joint Entropy and Information-Based Collaboration of Automated Learning Machines

We are working to develop automated intelligent agents, which can act an...

LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates

In cooperative multi-agent reinforcement learning (MARL), where agents o...

Learning to Share and Hide Intentions using Information Regularization

Learning to cooperate with friends and compete with foes is a key compon...

Multi-agent Collaboration for Feasible Collaborative Behavior Construction and Evaluation

In the case of the two-person zero-sum stochastic game with a central co...

A reinforcement learning algorithm for building collaboration in multi-agent systems

This paper presents a proof-of concept study for demonstrating the viabi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.