Stochastic Coordinate Minimization with Progressive Precision for Stochastic Convex Optimization

by   Sudeep Salgia, et al.

A framework based on iterative coordinate minimization (CM) is developed for stochastic convex optimization. Given that exact coordinate minimization is impossible due to the unknown stochastic nature of the objective function, the crux of the proposed optimization algorithm is an optimal control of the minimization precision in each iteration. We establish the optimal precision control and the resulting order-optimal regret performance for strongly convex and separably nonsmooth functions. An interesting finding is that the optimal progression of precision across iterations is independent of the low-dimensional CM routine employed, suggesting a general framework for extending low-dimensional optimization routines to high-dimensional problems. The proposed algorithm is amenable to online implementation and inherits the scalability and parallelizability properties of CM for large-scale optimization. Requiring only a sublinear order of message exchanges, it also lends itself well to distributed computing as compared with the alternative approach of coordinate gradient descent.


page 1

page 2

page 3

page 4


Derivation of Coordinate Descent Algorithms from Optimal Control Theory

Recently, it was posited that disparate optimization algorithms may be c...

Fast stochastic dual coordinate descent algorithms for linearly constrained convex optimization

The problem of finding a solution to the linear system Ax = b with certa...

Algorithmic Connections Between Active Learning and Stochastic Convex Optimization

Interesting theoretical associations have been established by recent pap...

DISCOMAX: A Proximity-Preserving Distance Correlation Maximization Algorithm

In a regression setting we propose algorithms that reduce the dimensiona...

Variational Principles for Mirror Descent and Mirror Langevin Dynamics

Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a pr...

An Analysis of Asynchronous Stochastic Accelerated Coordinate Descent

Gradient descent, and coordinate descent in particular, are core tools i...

A Coordinate-wise Optimization Algorithm for Sparse Inverse Covariance Selection

Sparse inverse covariance selection is a fundamental problem for analyzi...

Please sign up or login with your details

Forgot password? Click here to reset