Conditional Accelerated Lazy Stochastic Gradient Descent

03/16/2017

∙

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate O(1/ε^2) improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan and Kale [2012] with convergence rate O(1/ε^4).

READ FULL TEXT

Conditional Accelerated Lazy Stochastic Gradient Descent

A Sharp Convergence Rate for the Asynchronous Stochastic Gradient Descent

Stochastic gradient descent algorithms for strongly convex functions at O(1/T) convergence rates

Asynchronous decentralized accelerated stochastic gradient descent

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Asymptotically efficient one-step stochastic gradient descent

Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime

Stochastic Gradient Coding for Straggler Mitigation in Distributed Learning

Conditional Accelerated Lazy Stochastic Gradient Descent

Related Research

A Sharp Convergence Rate for the Asynchronous Stochastic Gradient Descent

Stochastic gradient descent algorithms for strongly convex functions at O(1/T) convergence rates

Asynchronous decentralized accelerated stochastic gradient descent

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Asymptotically efficient one-step stochastic gradient descent

Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime

Stochastic Gradient Coding for Straggler Mitigation in Distributed Learning