research
∙
11/15/2022
Mechanistic Mode Connectivity
Neural networks are known to be biased towards learning mechanisms that ...
share
research
∙
10/02/2022
What shapes the loss landscape of self-supervised learning?
Prevention of complete and dimensional collapse of representations has r...
research
∙
07/19/2021
Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
In this work we explore the limiting dynamics of deep neural networks tr...
research
∙
06/10/2021
Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
Inspired by BatchNorm, there has been an explosion of normalization laye...
research
∙
05/06/2021
Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning
In nature, symmetry governs regularities, while symmetry breaking brings...
research
∙
12/08/2020
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Predicting the dynamics of neural network parameters during training is ...
research
∙
06/09/2020
Pruning neural networks without any data by iteratively conserving synaptic flow
Pruning the parameters of deep neural networks has generated intense int...
research
∙
12/12/2019