research
∙
05/31/2022
Feature Learning in L_2-regularized DNNs: Attraction/Repulsion and Sparsity
We study the loss surface of DNNs with L_2 regularization. We show that ...
share
research
∙
06/30/2021
Deep Linear Networks Dynamics: Low-Rank Biases Induced by Initialization Scale and L2 Regularization
For deep linear networks (DLN), various hyperparameters alter the dynami...
research
∙
05/25/2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
We study how permutation symmetries in overparameterized multi-layer neu...
research
∙
02/05/2021
Smart Proofs via Smart Contracts: Succinct and Informative Mathematical Derivations via Decentralized Markets
Modern mathematics is built on the idea that proofs should be translatab...
research
∙
06/17/2020
Kernel Alignment Risk Estimator: Risk Prediction from Training Data
We study the risk (i.e. generalization error) of Kernel Ridge Regression...
research
∙
02/19/2020
Implicit Regularization of Random Feature Models
Random Feature (RF) models are used as efficient parametric approximatio...
research
∙
10/01/2019
The asymptotic spectrum of the Hessian of DNN throughout training
The dynamics of DNNs during gradient descent is described by the so-call...
research
∙
07/11/2019
Freeze and Chaos for DNNs: an NTK view of Batch Normalization, Checkerboard and Boundary Effects
In this paper, we analyze a number of architectural features of Deep Neu...
research
∙
01/06/2019
Scaling description of generalization with number of parameters in deep learning
We provide a description for the evolution of the generalization perform...
research
∙
06/20/2018