Connectivity-driven Communication in Multi-agent Reinforcement Learning through Diffusion Processes on Graphs

02/12/2020
by   Emanuele Pesce, et al.
9

We discuss the problem of learning collaborative behaviour in multi-agent systems using deep reinforcement learning (DRL). A connectivity-driven communication (CDC) algorithm is proposed to address three key aspects: what agents to involve in the communication, what information content to share, and how often to share it. We introduce the notion of a connectivity network, modelled as a weighted graph, where nodes represent agents and edges represent the degree of connectivity between pairs of agents. The optimal graph topology is learned end-to-end concurrently with the stochastic policy so as to maximise future expected returns. The communication patterns depend on the graph's topology through a diffusion process on the graph, the heat kernel, which is found by exponentiating the Laplacian eigensystem through time and is fully differentiable. Empirical results show that CDC is capable of superior performance over alternative algorithms for a range of cooperative navigation tasks.

READ FULL TEXT

page 7

page 13

page 14

page 17

page 18

research
09/12/2019

Modeling Sensorimotor Coordination as Multi-Agent Reinforcement Learning with Differentiable Communication

Multi-agent reinforcement learning has shown promise on a variety of coo...
research
08/22/2022

Exploring Task-oriented Communication in Multi-agent System: A Deep Reinforcement Learning Approach

The multi-agent system (MAS) enables the sharing of capabilities among a...
research
04/14/2020

On the Optimal Interaction Range for Multi-Agent Systems Under Adversarial Attack

Consider a consensus-driven multi-agent dynamic system. The interaction ...
research
01/12/2019

Improving Coordination in Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

Deep reinforcement learning algorithms have recently been used to train ...
research
02/16/2019

Communication Topologies Between Learning Agents in Deep Reinforcement Learning

A common technique to improve speed and robustness of learning in deep r...
research
11/04/2022

Emergent Quantized Communication

The field of emergent communication aims to understand the characteristi...
research
06/20/2023

Cooperative Multi-Agent Learning for Navigation via Structured State Abstraction

Cooperative multi-agent reinforcement learning (MARL) for navigation ena...

Please sign up or login with your details

Forgot password? Click here to reset