research
∙
03/22/2022
Practical tradeoffs between memory, compute, and performance in learned optimizers
Optimization plays a costly and crucial role in developing machine learn...
share
research
∙
10/28/2021
Understanding How Encoder-Decoder Architectures Attend
Encoder-decoder networks with attention have proven to be a powerful way...
research
∙
01/14/2021
Training Learned Optimizers with Randomly Initialized Learned Optimizers
Learned optimizers are increasingly effective, with performance exceedin...
research
∙
11/04/2020
Reverse engineering learned optimizers reveals known and novel mechanisms
Learned optimizers are algorithms that can themselves be trained to solv...
research
∙
10/28/2020
The geometry of integration in text classification RNNs
Despite the widespread application of recurrent neural networks (RNNs) a...
research
∙
09/23/2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
Much as replacing hand-designed features with learned functions has revo...
research
∙
04/17/2020
How recurrent networks implement contextual processing in sentiment analysis
Neural networks have a remarkable capacity for contextual processing–usi...
research
∙
02/27/2020
Using a thousand optimization tasks to learn hyperparameter search strategies
We present TaskSet, a dataset of tasks for use in training and evaluatin...
research
∙
12/12/2019
From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction
Recently, deep feedforward neural networks have achieved considerable su...
research
∙
07/19/2019
Universality and individuality in neural dynamics across large populations of recurrent networks
Task-based modeling with recurrent neural networks (RNNs) has emerged as...
research
∙
06/25/2019
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
Recurrent neural networks (RNNs) are a widely used tool for modeling seq...
research
∙
06/08/2019
Using learned optimizers to make models robust to input noise
State-of-the art vision models can achieve superhuman performance on ima...
research
∙
10/24/2018
Learned optimizers that outperform SGD on wall-clock and test loss
Deep learning has shown that learned functions can dramatically outperfo...
research
∙
10/24/2018
Learned optimizers that outperform SGD on wall-clock and validation loss
Deep learning has shown that learned functions can dramatically outperfo...
research
∙
06/26/2018
Guided evolutionary strategies: escaping the curse of dimensionality in random search
Many applications in machine learning require optimizing a function whos...
research
∙
03/31/2018
Learning Unsupervised Learning Rules
A major goal of unsupervised learning is to discover data representation...
research
∙
11/28/2017
Recurrent Segmentation for Variable Computational Budgets
State-of-the-art systems for semantic image segmentation utilize feed-fo...
research
∙
03/14/2017
Learned Optimizers that Scale and Generalize
Learning to learn has emerged as an important direction for achieving ar...
research
∙
02/06/2017
Deep Learning Models of the Retinal Response to Natural Scenes
A central challenge in neuroscience is to understand neural computations...
research
∙
03/12/2015