Exponential convergence of testing error for stochastic gradient methods

12/13/2017

∙

We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing error (classification error) converges exponentially fast if low-noise conditions are assumed.

READ FULL TEXT

Exponential convergence of testing error for stochastic gradient methods

Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors

Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features

On the stability of the stochastic gradient Langevin algorithm with dependent data stream

Can speed up the convergence rate of stochastic gradient methods to O(1/k^2) by a gradient averaging strategy?

On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators

Statistical Hypothesis Testing Based on Machine Learning: Large Deviations Analysis

Exponentially convergent stochastic k-PCA without variance reduction

Exponential convergence of testing error for stochastic gradient methods

Related Research

Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors

Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features

On the stability of the stochastic gradient Langevin algorithm with dependent data stream

Can speed up the convergence rate of stochastic gradient methods to O(1/k^2) by a gradient averaging strategy?

On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators

Statistical Hypothesis Testing Based on Machine Learning: Large Deviations Analysis

Exponentially convergent stochastic k-PCA without variance reduction