
Probabilistic analysis of Wiedemann's algorithm for minimal polynomial computation
Blackbox algorithms for linear algebra problems start with projection of...
read it

The Lanczos Algorithm Under Few Iterations: Concentration and Location of the Ritz Values
We study the Lanczos algorithm where the initial vector is sampled unifo...
read it

On TD(0) with function approximation: Concentration bounds and a centered variant with exponential convergence
We provide nonasymptotic bounds for the wellknown temporal difference ...
read it

Linear and Fisher Separability of Random Points in the ddimensional Spherical Layer
Stochastic separation theorems play important role in highdimensional d...
read it

Approximating the Permanent by Sampling from Adaptive Partitions
Computing the permanent of a nonnegative matrix is a core problem with ...
read it

Robust Bounds on Choosing from Large Tournaments
Tournament solutions provide methods for selecting the "best" alternativ...
read it

Bootstrapping the error of Oja's Algorithm
We consider the problem of quantifying uncertainty for the estimation er...
read it
Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize
This paper provides a nonasymptotic analysis of linear stochastic approximation (LSA) algorithms with fixed stepsize. This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system A̅θ = b̅ for which A̅ and b̅ can only be accessed through random estimates {( A_n, b_n): n ∈ℕ^*}. Our analysis is based on new results regarding moments and high probability bounds for products of matrices which are shown to be tight. We derive high probability bounds on the performance of LSA under weaker conditions on the sequence {( A_n, b_n): n ∈ℕ^*} than previous works. However, in contrast, we establish polynomial concentration bounds with order depending on the stepsize. We show that our conclusions cannot be improved without additional assumptions on the sequence of random matrices { A_n: n ∈ℕ^*}, and in particular that no Gaussian or exponential high probability bounds can hold. Finally, we pay a particular attention to establishing bounds with sharp order with respect to the number of iterations and the stepsize and whose leading terms contain the covariance matrices appearing in the central limit theorems.
READ FULL TEXT
Comments
There are no comments yet.