research
∙
06/22/2022
A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta
Mini-batch SGD with momentum is a fundamental algorithm for learning lar...
share
research
∙
02/24/2022
Embedded Ensembles: Infinite Width Limit and Operating Regimes
A memory efficient approach to ensembling neural networks is to share mo...
research
∙
02/02/2022
Tight Convergence Rate Bounds for Optimization Under Power Law Spectral Conditions
Performance of optimization on quadratic problems sensitively depends on...
research
∙
05/02/2021
Universal scaling laws in the gradient descent training of neural networks
Current theoretical results on optimization trajectories of neural netwo...
research
∙
02/22/2021
Elementary superexpressive activations
We call a finite family of activation functions superexpressive if any m...
research
∙
08/03/2020
Low-loss connection of weight vectors: distribution-based approaches
Recent research shows that sublevel sets of the loss surfaces of overpar...
research
∙
03/20/2020
High Performance Interference Suppression in Multi-User Massive MIMO Detector
In this paper, we propose a new nonlinear detector with improved interfe...
research
∙
06/22/2019
The phase diagram of approximation rates for deep neural networks
We explore the phase diagram of approximation rates for deep neural netw...
research
∙
10/09/2018
Collective evolution of weights in wide neural networks
We derive a nonlinear integro-differential transport equation describing...
research
∙
04/26/2018
Universal approximations of invariant maps by neural networks
We describe generalizations of the universal approximation theorem for n...
research
∙
02/10/2018
Optimal approximation of continuous functions by very deep ReLU networks
We prove that deep ReLU neural networks with conventional fully-connecte...
research
∙
05/03/2017
Quantified advantage of discontinuous weight selection in approximations with deep neural networks
We consider approximations of 1D Lipschitz functions by deep ReLU networ...
research
∙
01/16/2017
Geometric features for voxel-based surface recognition
We introduce a library of geometric voxel features for CAD surface recog...
research
∙
10/03/2016
Error bounds for approximations with deep ReLU networks
We study expressive power of shallow and deep neural networks with piece...
research
∙
09/05/2016