A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

by   Tze-Yang Tung, et al.

We propose a novel formulation of the "effectiveness problem" in communications, put forth by Shannon and Weaver in their seminal work [2], by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment can also communicate with each other over a noisy communication channel. The noisy communication channel is considered explicitly as part of the dynamics of the environment and the message each agent sends is part of the action that the agent can take. As a result, the agents learn not only to collaborate with each other but also to communicate "effectively" over a noisy channel. This framework generalizes both the traditional communication problem, where the main goal is to convey a message reliably over a noisy channel, and the "learning to communicate" framework that has received recent attention in the MARL literature, where the underlying communication channels are assumed to be error-free. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately from the underlying MA-POMDP. This is a very powerful framework, which has many real world applications, from autonomous vehicle planning to drone swarm control, and opens up the rich toolbox of deep reinforcement learning for the design of multi-user communication systems.


On Learning How to Communicate Over Noisy Channels for Collaborative Tasks

Consider a collaborative task carried out by two autonomous agents that ...

Control and Coordination of a SWARM of Unmanned Surface Vehicles using Deep Reinforcement Learning in ROS

An unmanned surface vehicle (USV) can perform complex missions by contin...

Multi-Agent Intention Sharing via Leader-Follower Forest

Intention sharing is crucial for efficient cooperation under partially o...

Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module

Designing an effective communication mechanism among agents in reinforce...

Low Entropy Communication in Multi-Agent Reinforcement Learning

Communication in multi-agent reinforcement learning has been drawing att...

Frequency-Based Patrolling with Heterogeneous Agents and Limited Communication

This paper investigates multi-agent frequencybased patrolling of interse...

Over-communicate no more: Situated RL agents learn concise communication protocols

While it is known that communication facilitates cooperation in multi-ag...

Please sign up or login with your details

Forgot password? Click here to reset