Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation

05/10/2023
∙
by   Yifei Min, et al.
∙
3
∙

We study multi-agent reinforcement learning in the setting of episodic Markov decision processes, where multiple agents cooperate via communication through a central server. We propose a provably efficient algorithm based on value iteration that enable asynchronous communication while ensuring the advantage of cooperation with low communication overhead. With linear function approximation, we prove that our algorithm enjoys an 𝒊Ėƒ(d^3/2H^2√(K)) regret with 𝒊Ėƒ(dHM^2) communication complexity, where d is the feature dimension, H is the horizon length, M is the total number of agents, and K is the total number of episodes. We also provide a lower bound showing that a minimal ÎĐ(dM) communication complexity is required to improve the performance through collaboration.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
∙ 07/07/2022

A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits

We study federated contextual linear bandits, where M agents cooperate w...
research
∙ 02/22/2021

Communication Efficient Parallel Reinforcement Learning

We consider the problem where M agents interact with M identical and ind...
research
∙ 03/08/2021

Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation

Reinforcement learning in cooperative multi-agent settings has recently ...
research
∙ 02/27/2020

A Visual Communication Map for Multi-Agent Deep Reinforcement Learning

Multi-agent learning distinctly poses significant challenges in the effo...
research
∙ 12/09/2019

Optimism in Reinforcement Learning with Generalized Linear Function Approximation

We design a new provably efficient algorithm for episodic reinforcement ...
research
∙ 06/01/2023

Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learning

Fairness plays a crucial role in various multi-agent systems (e.g., comm...
research
∙ 02/08/2023

Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learning

A practical challenge in reinforcement learning are combinatorial action...

Please sign up or login with your details

Forgot password? Click here to reset