DM^2: Distributed Multi-Agent Reinforcement Learning for Distribution Matching

06/01/2022
by   Caroline Wang, et al.
0

Current approaches to multi-agent cooperation rely heavily on centralized mechanisms or explicit communication protocols to ensure convergence. This paper studies the problem of distributed multi-agent learning without resorting to explicit coordination schemes. The proposed algorithm (DM^2) leverages distribution matching to facilitate independent agents' coordination. Each individual agent matches a target distribution of concurrently sampled trajectories from a joint expert policy. The theoretical analysis shows that under some conditions, if each agent optimizes their individual distribution matching objective, the agents increase a lower bound on the objective of matching the joint expert policy, allowing convergence to the joint expert policy. Further, if the distribution matching objective is aligned with a joint task, a combination of environment reward and distribution matching reward leads to the same equilibrium. Experimental validation on the StarCraft domain shows that combining the reward for distribution matching with the environment reward allows agents to outperform a fully distributed baseline. Additional experiments probe the conditions under which expert demonstrations need to be sampled in order to outperform the fully distributed baseline.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2019

CESMA: Centralized Expert Supervises Multi-Agents

We consider the reinforcement learning problem of training multiple agen...
research
04/15/2021

Joint Attention for Multi-Agent Coordination and Social Learning

Joint attention - the ability to purposefully coordinate attention with ...
research
05/31/2019

Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning

Many potential applications of reinforcement learning in the real world ...
research
09/14/2021

DSDF: An approach to handle stochastic agents in collaborative multi-agent reinforcement learning

Multi-Agent reinforcement learning has received lot of attention in rece...
research
09/17/2022

Sample-Efficient Multi-Agent Reinforcement Learning with Demonstrations for Flocking Control

Flocking control is a significant problem in multi-agent systems such as...
research
11/02/2020

Multi-Agent Reinforcement Learning for Persistent Monitoring

The Persistent Monitoring (PM) problem seeks to find a set of trajectori...
research
10/29/2019

Distributed and Consistent Multi-Image Feature Matching via QuickMatch

In this work we consider the multi-image object matching problem, extend...

Please sign up or login with your details

Forgot password? Click here to reset