research
∙
06/30/2023
The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit
In deep learning theory, the covariance matrix of the representations se...
share
research
∙
02/20/2023
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation
Skip connections and normalisation layers form two standard architectura...
research
∙
10/22/2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning
We study an approach to learning pruning masks by optimizing the expecte...
research
∙
10/24/2020
Stable ResNet
Deep ResNet architectures have achieved state of the art performance on ...
research
∙
07/11/2020
