research
∙
04/21/2023
Noise-Reuse in Online Evolution Strategies
Online evolution strategies have become an attractive alternative to aut...
share
research
∙
12/08/2022
General-Purpose In-Context Learning by Meta-Learning Transformers
Modern machine learning requires system designers to specify aspects of ...
research
∙
12/02/2022
Transformer-Based Learned Optimization
In this paper, we propose a new approach to learned optimization. As com...
research
∙
11/17/2022
VeLO: Training Versatile Learned Optimizers by Scaling Up
While deep learning models have replaced hand-designed features across m...
research
∙
10/11/2022
Discovered Policy Optimisation
Tremendous progress has been made in reinforcement learning (RL) over th...
research
∙
09/22/2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases
Learned optimizers – neural networks that are trained to act as optimize...
research
∙
03/22/2022
Practical tradeoffs between memory, compute, and performance in learned optimizers
Optimization plays a costly and crucial role in developing machine learn...
research
∙
12/27/2021
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies
Unrolled computation graphs arise in many scenarios, including training ...
research
∙
12/24/2021
Lyapunov Exponents for Diversity in Differentiable Games
Ridge Rider (RR) is an algorithm for finding diverse solutions to optimi...
research
∙
11/10/2021
Gradients are Not All You Need
Differentiable programming techniques are widely used in the community a...
research
∙
07/20/2021
Learn2Hop: Learned Optimization on Rough Landscapes
Optimization of non-convex loss surfaces containing many local minima re...
research
∙
01/14/2021
Training Learned Optimizers with Randomly Initialized Learned Optimizers
Learned optimizers are increasingly effective, with performance exceedin...
research
∙
12/07/2020
Parallel Training of Deep Networks with Local Updates
Deep learning models trained on large data sets have been widely success...
research
∙
11/12/2020
Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian
Over the last decade, a single algorithm has changed many facets of our ...
research
∙
11/04/2020
Reverse engineering learned optimizers reveals known and novel mechanisms
Learned optimizers are algorithms that can themselves be trained to solv...
research
∙
09/23/2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
Much as replacing hand-designed features with learned functions has revo...
research
∙
07/01/2020
On Linear Identifiability of Learned Representations
Identifiability is a desirable property of a statistical model: it impli...
research
∙
02/27/2020
Using a thousand optimization tasks to learn hyperparameter search strategies
We present TaskSet, a dataset of tasks for use in training and evaluatin...
research
∙
01/10/2020
Towards GAN Benchmarks Which Require Generalization
For many evaluation metrics commonly used as benchmarks for unconditiona...
research
∙
10/29/2019
Learning to Predict Without Looking Ahead: World Models Without Forward Prediction
Much of model-based reinforcement learning involves learning a model of ...
research
∙
09/20/2019
Learning an Adaptive Learning Rate Schedule
The learning rate is one of the most important hyper-parameters for mode...
research
∙
06/08/2019
Using learned optimizers to make models robust to input noise
State-of-the art vision models can achieve superhuman performance on ima...
research
∙
10/24/2018
Learned optimizers that outperform SGD on wall-clock and test loss
Deep learning has shown that learned functions can dramatically outperfo...
research
∙
10/24/2018
Learned optimizers that outperform SGD on wall-clock and validation loss
Deep learning has shown that learned functions can dramatically outperfo...
research
∙
06/26/2018
Guided evolutionary strategies: escaping the curse of dimensionality in random search
Many applications in machine learning require optimizing a function whos...
research
∙
03/31/2018
Learning Unsupervised Learning Rules
A major goal of unsupervised learning is to discover data representation...
research
∙
01/09/2018
Adversarial Spheres
State of the art computer vision models have been shown to be vulnerable...
research
∙
05/14/2017
Discrete Sequential Prediction of Continuous Actions for Deep RL
It has long been assumed that high dimensional continuous control proble...
research
∙
03/31/2017
BEGAN: Boundary Equilibrium Generative Adversarial Networks
We propose a new equilibrium enforcing method paired with a loss derived...
research
∙
11/07/2016