research
∙
07/18/2023
How Many Neurons Does it Take to Approximate the Maximum?
We study the size of a neural network needed to approximate the maximum ...
share
research
∙
05/18/2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias
We study the dynamics and implicit bias of gradient flow (GF) on univari...
research
∙
12/04/2021
Optimization-Based Separations for Neural Networks
Depth separation results propose a possible theoretical explanation for ...
research
∙
06/12/2021
Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems
Recently, there has been much interest in studying the convergence rates...
research
∙
06/01/2020
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks
We study the effects of mild over-parameterization on the optimization l...
research
∙
07/31/2019
How Good is SGD with Random Shuffling?
We study the performance of stochastic gradient descent (SGD) on smooth ...
research
∙
04/15/2019
Depth Separations in Neural Networks: What is Actually Being Separated?
Existing depth separation results for constant-depth networks essentiall...
research
∙
01/30/2019
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
The existence of adversarial examples in which an imperceptible change i...
research
∙
12/24/2017
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks
We consider the optimization problem associated with training simple ReL...
research
∙
10/31/2016
Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks
We provide several new depth-based separation results for feed-forward n...
research
∙
11/13/2015
