Convergence rates for the stochastic gradient descent method for non-convex objective functions

04/02/2019

∙

We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are applicable to simple objective functions arising in machine learning.

READ FULL TEXT

Convergence rates for the stochastic gradient descent method for non-convex objective functions

On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes

Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD

Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions

Convergence rates for momentum stochastic gradient descent with noise of machine learning type

Convergence of stochastic gradient descent under a local Lajasiewicz condition for deep neural networks

A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima

Sub-linear convergence of a tamed stochastic gradient descent method in Hilbert space

Convergence rates for the stochastic gradient descent method for non-convex objective functions

Related Research

On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes

Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD

Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions

Convergence rates for momentum stochastic gradient descent with noise of machine learning type

Convergence of stochastic gradient descent under a local Lajasiewicz condition for deep neural networks

A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima

Sub-linear convergence of a tamed stochastic gradient descent method in Hilbert space