-
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering
Standard first-order stochastic optimization algorithms base their updat...
11/09/2020 ∙ by Ricky T. Q. Chen, et al. ∙
0
∙
share
read it
-
Limitations of the Empirical Fisher Approximation
Natural gradient descent, which preconditions a gradient descent update ...
05/29/2019 ∙ by Frederik Kunstner, et al. ∙
0
∙
share
read it
-
DeepOBS: A Deep Learning Optimizer Benchmark Suite
Because the choice and tuning of the optimizer affects the speed, and ul...
03/13/2019 ∙ by Frank Schneider, et al. ∙
0
∙
share
read it
-
Follow the Signs for Robust Stochastic Optimization
Stochastic noise on gradients is now a common feature in machine learnin...
05/22/2017 ∙ by Lukas Balles, et al. ∙
0
∙
share
read it
-
Early Stopping without a Validation Set
Early stopping is a widely used technique to prevent poor generalization...
03/28/2017 ∙ by Maren Mahsereci, et al. ∙
0
∙
share
read it
-
Coupling Adaptive Batch Sizes with Learning Rates
Mini-batch stochastic gradient descent and variants thereof have become ...
12/15/2016 ∙ by Lukas Balles, et al. ∙
0
∙
share
read it
