Log In Sign Up

Connectivity-driven Communication in Multi-agent Reinforcement Learning through Diffusion Processes on Graphs

by   Emanuele Pesce, et al.

We discuss the problem of learning collaborative behaviour in multi-agent systems using deep reinforcement learning (DRL). A connectivity-driven communication (CDC) algorithm is proposed to address three key aspects: what agents to involve in the communication, what information content to share, and how often to share it. We introduce the notion of a connectivity network, modelled as a weighted graph, where nodes represent agents and edges represent the degree of connectivity between pairs of agents. The optimal graph topology is learned end-to-end concurrently with the stochastic policy so as to maximise future expected returns. The communication patterns depend on the graph's topology through a diffusion process on the graph, the heat kernel, which is found by exponentiating the Laplacian eigensystem through time and is fully differentiable. Empirical results show that CDC is capable of superior performance over alternative algorithms for a range of cooperative navigation tasks.


page 7

page 13

page 14

page 17

page 18


Modeling Sensorimotor Coordination as Multi-Agent Reinforcement Learning with Differentiable Communication

Multi-agent reinforcement learning has shown promise on a variety of coo...

On the Optimal Interaction Range for Multi-Agent Systems Under Adversarial Attack

Consider a consensus-driven multi-agent dynamic system. The interaction ...

Exploring Task-oriented Communication in Multi-agent System: A Deep Reinforcement Learning Approach

The multi-agent system (MAS) enables the sharing of capabilities among a...

Communication Topologies Between Learning Agents in Deep Reinforcement Learning

A common technique to improve speed and robustness of learning in deep r...

Correcting Experience Replay for Multi-Agent Communication

We consider the problem of learning to communicate using multi-agent rei...

Neurosymbolic Transformers for Multi-Agent Communication

We study the problem of inferring communication structures that can solv...