research
∙
05/23/2023
Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning
We propose a new per-layer adaptive step-size procedure for stochastic f...
share
research
∙
02/08/2022
A Mini-Block Natural Gradient Method for Deep Neural Networks
The training of deep neural networks (DNNs) is currently predominantly d...
research
∙
06/05/2021
Tensor Normal Training for Deep Learning Models
Despite the predominant use of first-order methods for training deep lea...
research
∙
02/12/2021
Kronecker-factored Quasi-Newton Methods for Convolutional Neural Networks
Second-order methods have the capability of accelerating optimization by...
research
∙
06/16/2020
Practical Quasi-Newton Methods for Training Deep Neural Networks
We consider the development of practical stochastic quasi-Newton, and in...
research
∙
12/31/2019
A Dynamic Sampling Adaptive-SGD Method for Machine Learning
We propose a stochastic optimization method for minimizing loss function...
research
∙
06/05/2019
Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networks
We present practical Levenberg-Marquardt variants of Gauss-Newton and na...
research
∙
05/24/2019
Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models
We consider distributed optimization under communication constraints for...
research
∙
11/20/2017
Unbiased Simulation for Optimizing Stochastic Function Compositions
In this paper, we introduce an unbiased gradient simulation algorithms f...
research
∙
06/05/2017
Greedy Approaches to Symmetric Orthogonal Tensor Decomposition
Finding the symmetric and orthogonal decomposition (SOD) of a tensor is ...
research
∙
05/29/2017
Successive Rank-One Approximations for Nearly Orthogonally Decomposable Symmetric Tensors
Many idealized problems in signal processing, machine learning and stati...
research
∙
07/05/2016
Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
In this paper we study stochastic quasi-Newton methods for nonconvex sto...
research
∙
03/29/2014
Scalable Robust Matrix Recovery: Frank-Wolfe Meets Proximal Methods
Recovering matrices from compressive and grossly corrupted observations ...
research
∙
11/24/2013
Robust Low-rank Tensor Recovery: Models and Algorithms
Robust tensor recovery plays an instrumental role in robustifying tensor...
research
∙
07/22/2013
Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery
Recovering a low-rank tensor from incomplete information is a recurring ...
research
∙
05/04/2011
Structured Sparsity via Alternating Direction Methods
We consider a class of sparse learning problems in high dimensional feat...
research
∙
10/30/2010
Sparse Inverse Covariance Selection via Alternating Linearization Methods
Gaussian graphical models are of great interest in statistical learning....
research
∙
12/23/2009