DeepAI AI Chat
Log In Sign Up

Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games

by   Zhaoyi Zhou, et al.

We introduce a class of networked Markov potential games where agents are associated with nodes in a network. Each agent has its own local potential function, and the reward of each agent depends only on the states and actions of agents within a Κ-hop neighborhood. In this context, we propose a localized actor-critic algorithm. The algorithm is scalable since each agent uses only local information and does not need access to the global state. Further, the algorithm overcomes the curse of dimensionality through the use of function approximation. Our main results provide finite-sample guarantees up to a localization error and a function approximation error. Specifically, we achieve an 𝒊Ėƒ(Ïĩ^-4) sample complexity measured by the averaged Nash regret. This is the first finite-sample bound for multi-agent competitive games that does not depend on the number of agents.


page 1

page 2

page 3

page 4

∙ 05/26/2021

Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximation

In this paper, we develop a novel variant of off-policy natural actor-cr...
∙ 12/05/2019

Scalable Reinforcement Learning of Localized Policies for Multi-Agent Networked Systems

We study reinforcement learning (RL) in a setting with a network of agen...
∙ 06/11/2020

Distributed Reinforcement Learning in Multi-Agent Networked Systems

We study distributed reinforcement learning (RL) for a network of agents...
∙ 02/18/2021

Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm

In this paper, we provide finite-sample convergence guarantees for an of...
∙ 08/05/2021

Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach

One of the challenges for multi-agent reinforcement learning (MARL) is d...
∙ 11/15/2018

Seq2Seq Mimic Games: A Signaling Perspective

We study the emergence of communication in multiagent adversarial settin...
∙ 06/12/2022

Finite-Time Analysis of Fully Decentralized Single-Timescale Actor-Critic

Decentralized Actor-Critic (AC) algorithms have been widely utilized for...