A History of Meta-gradient: Gradient Methods for Meta-learning

02/20/2022

∙

The history of meta-learning methods based on gradient descent is reviewed, focusing primarily on methods that adapt step-size (learning rate) meta-parameters.

READ FULL TEXT

A History of Meta-gradient: Gradient Methods for Meta-learning

Meta Mirror Descent: Optimiser Learning for Fast Convergence

One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement Learning

Amortized Proximal Optimization

Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning

Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training

Meta Learning in the Continuous Time Limit

A History of Meta-gradient: Gradient Methods for Meta-learning

Related Research

Meta Mirror Descent: Optimiser Learning for Fast Convergence

One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement Learning

Amortized Proximal Optimization

Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning

Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training

Meta Learning in the Continuous Time Limit