Concentration of Contractive Stochastic Approximation and Reinforcement Learning

06/27/2021
by   Siddharth Chandak, et al.
0

Using a martingale concentration inequality, concentration bounds `from time n_0 on' are derived for stochastic approximation algorithms with contractive maps and both martingale difference and Markov noises. These are applied to reinforcement learning algorithms, in particular to asynchronous Q-learning and TD(0).

READ FULL TEXT

page 1

page 2

page 3

page 4

12/08/2020

A Concentration Inequality for the Facility Location Problem

We give a concentration inequality for a stochastic version of the facil...
09/18/2020

Deviation bound for non-causal machine learning

Concentration inequality are widely used for analysing machines learning...
05/12/2014

Sharp Finite-Time Iterated-Logarithm Martingale Concentration

We give concentration bounds for martingales that are uniform over finit...
09/08/2021

Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning

The stochastic approximation (SA) algorithm is a widely used probabilist...
08/02/2020

Concentration-Bound Analysis for Probabilistic Programs and Probabilistic Recurrence Relations

Analyzing probabilistic programs and randomized algorithms are classical...
04/09/2020

On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration

We undertake a precise study of the asymptotic and non-asymptotic proper...
11/13/2019

Improved Concentration Bounds for Gaussian Quadratic Forms

For a wide class of monotonic functions f, we develop a Chernoff-style c...