
(Non) asymptotic properties of Stochastic Gradient Langevin Dynamics
Applying standard Markov chain Monte Carlo (MCMC) algorithms to large da...
read it

An urn model with local reinforcement: a theoretical framework for a chisquared goodness of fit test with a big sample
Motivated by recent studies of big samples, this work aims at constructi...
read it

Linear Stochastic Approximation: Constant StepSize and Iterate Averaging
We consider ddimensional linear stochastic approximation algorithms (LS...
read it

Constant Step Size LeastMeanSquare: BiasVariance Tradeoffs and Optimal Sampling Distributions
We consider the leastsquares regression problem and provide a detailed ...
read it

On TD(0) with function approximation: Concentration bounds and a centered variant with exponential convergence
We provide nonasymptotic bounds for the wellknown temporal difference ...
read it

Refined Analysis of the Asymptotic Complexity of the Number Field Sieve
The classical heuristic complexity of the Number Field Sieve (NFS) is th...
read it

Two Timescale Stochastic Approximation with Controlled Markov noise and Offpolicy temporal difference learning
We present for the first time an asymptotic convergence analysis of two ...
read it
On Linear Stochastic Approximation: Finegrained PolyakRuppert and NonAsymptotic Concentration
We undertake a precise study of the asymptotic and nonasymptotic properties of stochastic approximation procedures with PolyakRuppert averaging for solving a linear system A̅θ = b̅. When the matrix A̅ is Hurwitz, we prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity. The CLT characterizes the exact asymptotic covariance matrix, which is the sum of the classical PolyakRuppert covariance and a correction term that scales with the step size. Under assumptions on the tail of the noise distribution, we prove a nonasymptotic concentration inequality whose main term matches the covariance in CLT in any direction, up to universal constants. When the matrix A̅ is not Hurwitz but only has nonnegative real parts in its eigenvalues, we prove that the averaged LSA procedure actually achieves an O(1/T) rate in meansquared error. Our results provide a more refined understanding of linear stochastic approximation in both the asymptotic and nonasymptotic settings. We also show various applications of the main results, including the study of momentumbased stochastic gradient methods as well as temporal difference algorithms in reinforcement learning.
READ FULL TEXT
Comments
There are no comments yet.