Meta-learners' learning dynamics are unlike learners'

05/03/2019
by   Neil C. Rabinowitz, et al.
26

Meta-learning is a tool that allows us to build sample-efficient learning systems. Here we show that, once meta-trained, LSTM Meta-Learners aren't just faster learners than their sample-inefficient deep learning (DL) and reinforcement learning (RL) brethren, but that they actually pursue fundamentally different learning trajectories. We study their learning dynamics on three sets of structured tasks for which the corresponding learning dynamics of DL and RL systems have been previously described: linear regression (Saxe et al., 2013), nonlinear regression (Rahaman et al., 2018; Xu et al., 2018), and contextual bandits (Schaul et al., 2019). In each case, while sample-inefficient DL and RL Learners uncover the task structure in a staggered manner, meta-trained LSTM Meta-Learners uncover almost all task structure concurrently, congruent with the patterns expected from Bayes-optimal inference algorithms. This has implications for research areas wherever the learning behaviour itself is of interest, such as safety, curriculum design, and human-in-the-loop machine learning.

READ FULL TEXT

page 7

page 8

page 9

page 20

research
02/19/2020

Curriculum in Gradient-Based Meta-Reinforcement Learning

Gradient-based meta-learners such as Model-Agnostic Meta-Learning (MAML)...
research
05/08/2019

Meta-learning of Sequential Strategies

In this report we review memory-based meta-learning as a tool for buildi...
research
04/07/2021

The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL

In this work, we analyze the reinstatement mechanism introduced by Ritte...
research
10/11/2021

REIN-2: Giving Birth to Prepared Reinforcement Learning Agents Using Reinforcement Learning Agents

Deep Reinforcement Learning (Deep RL) has been in the spotlight for the ...
research
02/04/2021

Alchemy: A structured task distribution for meta-reinforcement learning

There has been rapidly growing interest in meta-learning as a method for...
research
06/11/2018

Auto-Meta: Automated Gradient Based Meta Learner Search

Fully automating machine learning pipeline is one of the outstanding cha...
research
04/03/2017

Multi-Advisor Reinforcement Learning

We consider tackling a single-agent RL problem by distributing it to n l...

Please sign up or login with your details

Forgot password? Click here to reset