Log In Sign Up

Stochastic Mirror Descent for Large-Scale Sparse Recovery

by   Sasila Ilandarideva, et al.

In this paper we discuss an application of Stochastic Approximation to statistical estimation of high-dimensional sparse parameters. The proposed solution reduces to resolving a penalized stochastic optimization problem on each stage of a multistage algorithm; each problem being solved to a prescribed accuracy by the non-Euclidean Composite Stochastic Mirror Descent (CSMD) algorithm. Assuming that the problem objective is smooth and quadratically minorated and stochastic perturbations are sub-Gaussian, our analysis prescribes the method parameters which ensure fast convergence of the estimation error (the radius of a confidence ball of a given norm around the approximate solution). This convergence is linear during the first "preliminary" phase of the routine and is sublinear during the second "asymptotic" phase. We consider an application of the proposed approach to sparse Generalized Linear Regression problem. In this setting, we show that the proposed algorithm attains the optimal convergence of the estimation error under weak assumptions on the regressor distribution. We also present a numerical study illustrating the performance of the algorithm on high-dimensional simulation data.


page 1

page 2

page 3

page 4


Sparse recovery by reduced variance stochastic approximation

In this paper, we discuss application of iterative Stochastic Optimizati...

Composite Expectile Regression with Gene-environment Interaction

If error distribution has heteroscedasticity, it voliates the assumption...

Algorithms of Robust Stochastic Optimization Based on Mirror Descent Method

We propose an approach to construction of robust non-Euclidean iterative...

Sub-linear convergence of a stochastic proximal iteration method in Hilbert space

We consider a stochastic version of the proximal point algorithm for opt...

Signal recovery by Stochastic Optimization

We discuss an approach to signal recovery in Generalized Linear Models (...

Weak Collocation Regression method: fast reveal hidden stochastic dynamics from high-dimensional aggregate data

Revealing hidden dynamics from the stochastic data is a challenging prob...

Estimation of high dimensional Gamma convolutions through random projections

Multivariate generalized Gamma convolutions are distributions defined by...