Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learning

02/01/2020
by   Guannan Qu, et al.
0

We consider a general asynchronous Stochastic Approximation (SA) scheme featuring a weighted infinity-norm contractive operator, and prove a bound on its finite-time convergence rate on a single trajectory. Additionally, we specialize the result to asynchronous Q-learning. The resulting bound matches the sharpest available bound for synchronous Q-learning, and improves over previous known bounds for asynchronous Q-learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/10/2011

Asynchronous Stochastic Approximation with Differential Inclusions

The asymptotic pseudo-trajectory approach to stochastic approximation of...
research
04/05/2020

On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems

In the realm of big data and machine learning, data-parallel, distribute...
research
07/11/2023

Comparing Social Network Dynamic Operators

Numerous logics have been developed to reason either about threshold-ind...
research
09/08/2021

Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning

The stochastic approximation (SA) algorithm is a widely used probabilist...
research
10/17/2019

Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm Balls

Large-scale machine learning training suffers from two prior challenges,...
research
07/12/2023

Low complexity convergence rate bounds for the synchronous gossip subclass of push-sum algorithms

We develop easily accessible quantities for bounding the almost sure exp...
research
07/02/2019

Asynchronous Communications Library for the Parallel-in-Time Solution of Black-Scholes Equation

The advent of asynchronous iterative scheme gives high efficiency to num...

Please sign up or login with your details

Forgot password? Click here to reset