research
∙
06/13/2023
On Achieving Optimal Adversarial Test Error
We first elucidate various fundamental properties of optimal adversarial...
share
research
∙
06/05/2023
Representational Strengths and Limitations of Transformers
Attention layers, as commonly used in transformers, form the backbone of...
research
∙
08/04/2022
Feature selection with gradient descent on two-layer networks in low-rotation regimes
This work establishes low test error of gradient flow (GF) and stochasti...
research
∙
05/06/2022
Convex Analysis at Infinity: An Introduction to Astral Space
Not all convex functions on ℝ^n have finite minimizers; some can only be...
research
∙
02/14/2022
Stochastic linear optimization never overfits with quadratically-bounded losses on general data
This work shows that a diverse collection of linear optimization methods...
research
∙
10/21/2021
Actor-critic is implicitly biased towards high entropy optimal policies
We show that the simplest actor-critic method – a linear softmax policy ...
research
∙
07/01/2021
Fast Margin Maximization via Dual Acceleration
We present and analyze a momentum-based gradient method for training lin...
research
∙
06/10/2021
Early-stopped neural networks are consistent
This work studies the behavior of neural networks trained with the logis...
research
∙
04/12/2021
Generalization bounds via distillation
This paper theoretically investigates the following empirical phenomenon...
research
∙
06/19/2020
Gradient descent follows the regularization path for general losses
Recent work across many machine learning disciplines has highlighted tha...
research
∙
06/11/2020
Directional convergence and alignment in deep learning
In this paper, we show that although the minimizers of cross-entropy and...
research
∙
10/15/2019
Neural tangent kernels, transportation mappings, and universal approximation
This paper establishes rates of universal approximation for the shallow ...
research
∙
09/26/2019
Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks
Recent work has revealed that overparameterized networks trained by grad...
research
∙
06/18/2019
Approximation power of random neural networks
This paper investigates the approximation power of three types of random...
research
∙
06/11/2019
A refined primal-dual analysis of the implicit bias
Recent work shows that gradient descent on linearly separable data is im...
research
∙
06/08/2019
A gradual, semi-discrete approach to generative network training via explicit wasserstein minimization
This paper provides a simple procedure to fit generative networks to tar...
research
∙
10/26/2018
Size-Noise Tradeoffs in Generative Networks
This paper investigates the ability of generative networks to convert th...
research
∙
10/04/2018
Gradient descent aligns the layers of deep linear networks
This paper establishes risk convergence and asymptotic weight matrix ali...
research
∙
03/20/2018
Risk and parameter convergence of logistic regression
The logistic loss is strictly convex and does not attain its infimum; co...
research
∙
11/06/2017
Social Welfare and Profit Maximization from Revealed Preferences
Consider the seller's problem of finding "optimal" prices for her (divis...
research
∙
06/26/2017
Spectrally-normalized margin bounds for neural networks
This paper presents a margin-based multiclass generalization bound for n...
research
∙
06/11/2017
Neural networks and rational functions
Neural networks and rational functions efficiently approximate each othe...
research
∙
02/13/2017
Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
Stochastic Gradient Langevin Dynamics (SGLD) is a popular variant of Sto...
research
∙
02/14/2016
Benefits of depth in neural networks
For any positive integer k, there exist neural networks with Θ(k^3) laye...
research
∙
09/27/2015
Representation Benefits of Deep Feedforward Networks
This note provides a family of classification problems, indexed by a pos...
research
∙
06/15/2015
Convex Risk Minimization and Conditional Probability Estimation
This paper proves, in very general settings, that convex risk minimizati...
research
∙
10/02/2014
Scalable Nonlinear Learning with Adaptive Polynomial Expansions
Can we effectively learn a nonlinear representation in time comparable t...
research
∙
05/13/2013
Boosting with the Logistic Loss is Consistent
This manuscript provides optimization guarantees, generalization bounds,...
research
∙
03/18/2013
Margins, Shrinkage, and Boosting
This manuscript shows that AdaBoost and its immediate variants can produ...
research
∙
01/21/2013
Dirichlet draws are sparse with high probability
This note provides an elementary proof of the folklore fact that draws f...
research
∙
10/29/2012
Tensor decompositions for learning latent variable models
This work considers a computationally and statistically efficient parame...
research
∙
06/27/2012