research
∙
06/07/2023
Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks
In this work, we reveal a strong implicit bias of stochastic gradient de...
share
research
∙
10/07/2022
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks
In this work, we explore the maximum-margin bias of quasi-homogeneous ne...
research
∙
07/19/2021
Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
In this work we explore the limiting dynamics of deep neural networks tr...
research
∙
05/06/2021
Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning
In nature, symmetry governs regularities, while symmetry breaking brings...
research
∙
12/08/2020
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Predicting the dynamics of neural network parameters during training is ...
research
∙
06/09/2020
Pruning neural networks without any data by iteratively conserving synaptic flow
Pruning the parameters of deep neural networks has generated intense int...
research
∙
02/28/2020
Two Routes to Scalable Credit Assignment without Weight Symmetry
The neural plausibility of backpropagation has long been disputed, prima...
research
∙
01/23/2019