research
∙
06/16/2022
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence
A major challenge in modern machine learning is theoretically understand...
share
research
∙
04/06/2022
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
Contrastive learning is a highly effective method which uses unlabeled d...
research
∙
07/28/2021
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers
A common lens to theoretically study neural net architectures is to anal...
research
∙
06/17/2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning
Pretrained language models have achieved state-of-the-art performance wh...
research
∙
06/08/2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Recent works in self-supervised learning have advanced the state-of-the-...
research
∙
11/03/2020
Meta-learning Transferable Representations with a Single Target Domain
Recent works found that fine-tuning and joint training—two popular appro...
research
∙
10/07/2020
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
Self-training algorithms, which train a model to fit pseudolabels predic...
research
∙
06/17/2020
Self-training Avoids Using Spurious Features Under Domain Shift
In unsupervised domain adaptation, existing theory focuses on situations...
research
∙
06/15/2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance
The noise in stochastic gradient descent (SGD) provides a crucial implic...
research
∙
02/28/2020
The Implicit and Explicit Regularization Effects of Dropout
Dropout is a widely-used regularization technique, often required to obt...
research
∙
10/09/2019
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
For linear classifiers, the relationship between (normalized) output mar...
research
∙
07/10/2019
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Stochastic gradient descent with a large initial learning rate is a wide...
research
∙
06/18/2019
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
Deep learning algorithms can fare poorly when the training dataset suffe...
research
∙
05/09/2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
Existing Rademacher complexity bounds for neural networks rely only on n...
research
∙
10/12/2018
On the Margin Theory of Feedforward Neural Networks
Past works have shown that, somewhat surprisingly, over-parametrization ...
research
∙
10/15/2016