research
∙
04/21/2023
Prediction, Learning, Uniform Convergence, and Scale-sensitive Dimensions
We present a new general-purpose algorithm for learning classes of [0,1]...
share
research
∙
10/04/2022
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima
We consider Sharpness-Aware Minimization (SAM), a gradient-based optimiz...
research
∙
09/19/2022
Deep Linear Networks can Benignly Overfit when Shallow Ones Do
We bound the excess risk of interpolating deep linear networks trained u...
research
∙
12/08/2021
The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex loss
van Rooyen et al. introduced a notion of convex loss functions being rob...
research
∙
10/06/2021
Foolish Crowds Support Benign Overfitting
We prove a lower bound on the excess risk of sparse interpolating proced...
research
∙
08/25/2021
The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks
The recent success of neural network models has shone light on a rather ...
research
∙
05/21/2021
Properties of the After Kernel
The Neural Tangent Kernel (NTK) is the wide-network limit of a kernel de...
research
∙
02/09/2021
When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations?
We establish conditions under which gradient descent applied to fixed-wi...
research
∙
12/04/2020
When does gradient descent with logistic loss find interpolating two-layer networks?
We study the training of finite-width two-layer smoothed ReLU networks f...
research
∙
10/16/2020
Failures of model-dependent generalization bounds for least-norm interpolation
We consider bounds on the generalization performance of the least-norm l...
research
∙
04/25/2020
Finite-sample analysis of interpolating linear classifiers in the overparameterized regime
We prove bounds on the population risk of the maximum margin algorithm f...
research
∙
03/02/2020
On the Global Convergence of Training Deep Linear ResNets
We study the convergence of gradient descent (GD) and stochastic gradien...
research
∙
02/01/2020
Oracle lower bounds for stochastic gradient sampling algorithms
We consider the problem of sampling from a strongly log-concave density ...
research
∙
06/26/2019
Benign Overfitting in Linear Regression
The phenomenon of benign overfitting is one of the key mysteries uncover...
research
∙
05/29/2019
Size-free generalization bounds for convolutional neural networks
We prove bounds on the generalization error of convolutional networks. T...
research
∙
01/07/2019
On the effect of the activation function on the distribution of hidden nodes in a deep network
We analyze the joint probability distribution on the lengths of the vect...
research
∙
11/09/2018
Density estimation for shift-invariant multidimensional distributions
We study density estimation for classes of shift-invariant distributions...
research
∙
07/18/2018
Learning Sums of Independent Random Variables with Sparse Collective Support
We study the learnability of sums of independent integer random variable...
research
∙
05/26/2018
The Singular Values of Convolutional Layers
We characterize the singular values of the linear transformation associa...
research
∙
04/13/2018
Representing smooth functions as compositions of near-identity functions with implications for deep network optimization
We show that any smooth bi-Lipschitz h can be represented exactly as a c...
research
∙
02/16/2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks
We analyze algorithms for approximating a function f(x) = Φ x mapping ^d...
research
∙
02/14/2016
Surprising properties of dropout in deep networks
We analyze dropout in deep networks with rectified linear units and the ...
research
∙
12/15/2014
On the Inductive Bias of Dropout
Dropout is a simple but effective technique for learning in neural netwo...
research
∙
07/31/2013
The Power of Localization for Efficiently Learning Linear Separators with Noise
We introduce a new approach for designing computationally efficient lear...
research
∙
11/06/2012