-
Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime
We analyze the convergence of the averaged stochastic gradient descent f...
06/22/2020 ∙ by Atsushi Nitanda, et al. ∙
14
∙
share
read it
-
When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) oft...
06/18/2020 ∙ by Shun-ichi Amari, et al. ∙
0
∙
share
read it
-
Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features
Although kernel methods are widely used in many learning problems, they ...
11/13/2019 ∙ by Shingo Yashima, et al. ∙
12
∙
share
read it
-
Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
Deep learning has exhibited superior performance for various tasks, espe...
10/28/2019 ∙ by Taiji Suzuki, et al. ∙
11
∙
share
read it
-
Data Cleansing for Models Trained with SGD
Data cleansing is a typical approach used to improve the accuracy of mac...
06/20/2019 ∙ by Satoshi Hara, et al. ∙
3
∙
share
read it
-
Refined Generalization Analysis of Gradient Descent for Over-parameterized Two-layer Neural Networks with Smooth Activations on Classification Problems
Recently, several studies have proven the global convergence and general...
05/23/2019 ∙ by Atsushi Nitanda, et al. ∙
2
∙
share
read it
-
Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors
We consider stochastic gradient descent for binary classification proble...
06/14/2018 ∙ by Atsushi Nitanda, et al. ∙
0
∙
share
read it
-
Functional Gradient Boosting based on Residual Network Perception
Residual Networks (ResNets) have become state-of-the-art models in deep ...
02/25/2018 ∙ by Atsushi Nitanda, et al. ∙
0
∙
share
read it
-
Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Models
We propose a new technique that boosts the convergence of training gener...
01/07/2018 ∙ by Atsushi Nitanda, et al. ∙
0
∙
share
read it
-
Stochastic Particle Gradient Descent for Infinite Ensembles
The superior performance of ensemble methods with infinite models are we...
12/14/2017 ∙ by Atsushi Nitanda, et al. ∙
0
∙
share
read it
-
Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
We propose an optimization method for minimizing the finite sums of smoo...
06/09/2015 ∙ by Atsushi Nitanda, et al. ∙
0
∙
share
read it
