Universally Expressive Communication in Multi-Agent Reinforcement Learning

06/14/2022
by   Matthew Morris, et al.
0

Allowing agents to share information through communication is crucial for solving complex tasks in multi-agent reinforcement learning. In this work, we consider the question of whether a given communication protocol can express an arbitrary policy. By observing that many existing protocols can be viewed as instances of graph neural networks (GNNs), we demonstrate the equivalence of joint action selection to node labelling. With standard GNN approaches provably limited in their expressive capacity, we draw from existing GNN literature and consider augmenting agent observations with: (1) unique agent IDs and (2) random noise. We provide a theoretical analysis as to how these approaches yield universally expressive communication, and also prove them capable of targeting arbitrary sets of actions for identical agents. Empirically, these augmentations are found to improve performance on tasks where expressive communication is required, whilst, in general, the optimal communication protocol is found to be task-dependent.

READ FULL TEXT

page 36

page 37

page 38

page 39

page 42

research
12/14/2020

Specializing Inter-Agent Communication in Heterogeneous Multi-Agent Reinforcement Learning using Agent Class Information

Inspired by recent advances in agent communication with graph neural net...
research
11/14/2022

Dynamic Collaborative Multi-Agent Reinforcement Learning Communication for Autonomous Drone Reforestation

We approach autonomous drone-based reforestation with a collaborative mu...
research
01/12/2019

Improving Coordination in Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

Deep reinforcement learning algorithms have recently been used to train ...
research
02/11/2020

Learning Structured Communication for Multi-agent Reinforcement Learning

This work explores the large-scale multi-agent communication mechanism u...
research
01/06/2023

Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads

To integrate high amounts of renewable energy resources, electrical powe...
research
10/10/2018

Learning Multi-agent Implicit Communication Through Actions: A Case Study in Contract Bridge, a Collaborative Imperfect-Information Game

In situations where explicit communication is limited, a human collabora...

Please sign up or login with your details

Forgot password? Click here to reset