Local Stochastic Approximation: A Unified View of Federated Learning and Distributed Multi-Task Reinforcement Learning Algorithms

06/24/2020
by   Thinh T. Doan, et al.
0

Motivated by broad applications in reinforcement learning and federated learning, we study local stochastic approximation over a network of agents, where their goal is to find the root of an operator composed of the local operators at the agents. Our focus is to characterize the finite-time performance of this method when the data at each agent are generated from Markov processes, and hence they are dependent. In particular, we provide the convergence rates of local stochastic approximation for both constant and time-varying step sizes. Our results show that these rates are within a logarithmic factor of the ones under independent data. We then illustrate the applications of these results to different interesting problems in multi-task reinforcement learning and federated learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/28/2020

Finite-Time Analysis of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning

Stochastic approximation, a data-driven approach for finding the fixed p...
research
07/25/2019

Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation

We study the policy evaluation problem in multi-agent reinforcement lear...
research
03/11/2021

Multi-Task Federated Reinforcement Learning with Adversaries

Reinforcement learning algorithms, just like any other Machine learning ...
research
10/13/2022

Personalized Federated Hypernetworks for Privacy Preservation in Multi-Task Reinforcement Learning

Multi-Agent Reinforcement Learning currently focuses on implementations ...
research
12/23/2019

Finite-Time Analysis and Restarting Scheme for Linear Two-Time-Scale Stochastic Approximation

Motivated by their broad applications in reinforcement learning, we stud...
research
10/02/2020

Self-Play Reinforcement Learning for Fast Image Retargeting

In this study, we address image retargeting, which is a task that adjust...
research
03/24/2021

The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication

The paper considers a distributed version of deep reinforcement learning...

Please sign up or login with your details

Forgot password? Click here to reset