Minimizing Communication while Maximizing Performance in Multi-Agent Reinforcement Learning

06/15/2021
by   Varun Kumar Vijay, et al.
0

Inter-agent communication can significantly increase performance in multi-agent tasks that require co-ordination to achieve a shared goal. Prior work has shown that it is possible to learn inter-agent communication protocols using multi-agent reinforcement learning and message-passing network architectures. However, these models use an unconstrained broadcast communication model, in which an agent communicates with all other agents at every step, even when the task does not require it. In real-world applications, where communication may be limited by system constraints like bandwidth, power and network capacity, one might need to reduce the number of messages that are sent. In this work, we explore a simple method of minimizing communication while maximizing performance in multi-task learning: simultaneously optimizing a task-specific objective and a communication penalty. We show that the objectives can be optimized using Reinforce and the Gumbel-Softmax reparameterization. We introduce two techniques to stabilize training: 50 training and message forwarding. Training with the communication penalty on only 50 messages. Second, repeating messages received previously helps models retain information, and further improves performance. With these techniques, we show that we can reduce communication by 75

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2020

Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint

Communicating with each other in a distributed manner and behaving as a ...
research
02/10/2023

Low Entropy Communication in Multi-Agent Reinforcement Learning

Communication in multi-agent reinforcement learning has been drawing att...
research
02/28/2023

On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning

Explicit communication among humans is key to coordinating and learning....
research
10/02/2020

Correcting Experience Replay for Multi-Agent Communication

We consider the problem of learning to communicate using multi-agent rei...
research
11/30/2022

Towards True Lossless Sparse Communication in Multi-Agent Systems

Communication enables agents to cooperate to achieve their goals. Learni...
research
12/05/2016

Message Passing Multi-Agent GANs

Communicating and sharing intelligence among agents is an important face...
research
08/15/2019

Straggling for Covert Message Passing on Complete Graphs

We introduce a model for mobile, multi-agent information transfer that i...

Please sign up or login with your details

Forgot password? Click here to reset