-
Understanding the Role of Momentum in Stochastic Gradient Methods
The use of momentum in stochastic gradient methods has become a widespre...
10/30/2019 ∙ by Igor Gitman, et al. ∙
0
∙
share
read it
-
OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models
We present OpenSeq2Seq -- an open-source toolkit for training sequence-t...
05/25/2018 ∙ by Oleksii Kuchaiev, et al. ∙
0
∙
share
read it
-
Novel Prediction Techniques Based on Clusterwise Linear Regression
In this paper we explore different regression models based on Clusterwis...
04/28/2018 ∙ by Igor Gitman, et al. ∙
0
∙
share
read it
-
Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
The rise of deep learning in recent years has brought with it increasing...
01/09/2018 ∙ by Igor Gitman, et al. ∙
0
∙
share
read it
-
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification
Batch normalization (BN) has become a de facto standard for training dee...
09/24/2017 ∙ by Igor Gitman, et al. ∙
0
∙
share
read it
-
Large Batch Training of Convolutional Networks
A common way to speed up training of large convolutional networks is to ...
08/13/2017 ∙ by Yang You, et al. ∙
0
∙
share
read it
