research
∙
06/04/2021
Debiasing a First-order Heuristic for Approximate Bi-level Optimization
Approximate bi-level optimization (ABLO) consists of (outer-level) optim...
share
research
∙
12/21/2020
Sub-Linear Memory: How to Make Performers SLiM
The Transformer architecture has revolutionized deep learning on sequent...
research
∙
09/30/2020
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate re...
research
∙
06/05/2020
UFO-BLO: Unbiased First-Order Bilevel Optimization
Bilevel optimization (BLO) is a popular approach with many applications ...
research
∙
06/05/2020
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
Transformer models have achieved state-of-the-art results across a diver...
research
∙
04/18/2020
CWY Parametrization for Scalable Learning of Orthogonal and Stiefel Matrices
In this paper we propose a new approach for optimization over orthogonal...
research
∙
03/30/2020
