research
∙
06/09/2023
Using Foundation Models to Detect Policy Violations with Minimal Supervision
Foundation models, i.e. large neural networks pre-trained on large text ...
share
research
∙
08/03/2021
Large-Scale Differentially Private BERT
In this work, we study the large-scale pretraining of BERT-Large with di...
research
∙
02/20/2020
Second Order Optimization Made Practical
Optimization in machine learning, both theoretical and applied, is prese...
research
∙
01/30/2019
Memory-Efficient Adaptive Optimization for Large-Scale Learning
Adaptive gradient-based optimizers such as AdaGrad and Adam are among th...
research
∙
05/26/2018
The Singular Values of Convolutional Layers
We characterize the singular values of the linear transformation associa...
research
∙
02/26/2018
Shampoo: Preconditioned Stochastic Tensor Optimization
Preconditioned gradient methods are among the most general and powerful ...
research
∙
06/20/2017
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization
We describe a framework for deriving and analyzing online optimization a...
research
∙
11/19/2015