Decentralized Deterministic Multi-Agent Reinforcement Learning

02/19/2021
by   Antoine Grosnit, et al.
1

[Zhang, ICML 2018] provided the first decentralized actor-critic algorithm for multi-agent reinforcement learning (MARL) that offers convergence guarantees. In that work, policies are stochastic and are defined on finite action spaces. We extend those results to offer a provably-convergent decentralized actor-critic algorithm for learning deterministic policies on continuous action spaces. Deterministic policies are important in real-world settings. To handle the lack of exploration inherent in deterministic policies, we consider both off-policy and on-policy settings. We provide the expression of a local deterministic policy gradient, decentralized deterministic actor-critic algorithms and convergence guarantees for linearly-approximated value functions. This work will help enable decentralized MARL in high-dimensional action spaces and pave the way for more widespread use of MARL.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/03/2022

Asymptotic Convergence of Deep Multi-Agent Actor-Critic Algorithms

We present sufficient conditions that ensure convergence of the multi-ag...
research
10/05/2018

Actor-Attention-Critic for Multi-Agent Reinforcement Learning

Reinforcement learning in multi-agent scenarios is important for real-wo...
research
06/10/2019

Exploiting the sign of the advantage function to learn deterministic policies in continuous domains

In the context of learning deterministic policies in continuous domains,...
research
01/24/2019

Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots

State-of-the-art distributed algorithms for reinforcement learning rely ...
research
10/22/2022

Solving Continuous Control via Q-learning

While there has been substantial success in applying actor-critic method...
research
10/11/2021

Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees

Multi-agent reinforcement learning (MARL) has attracted much research at...
research
07/30/2023

Robust Multi-Agent Reinforcement Learning with State Uncertainty

In real-world multi-agent reinforcement learning (MARL) applications, ag...

Please sign up or login with your details

Forgot password? Click here to reset