DeepAI
Log In Sign Up

Meta Automatic Curriculum Learning

11/16/2020
by   Rémy Portelas, et al.
5

A major challenge in the Deep RL (DRL) community is to train agents able to generalize their control policy over situations never seen in training. Training on diverse tasks has been identified as a key ingredient for good generalization, which pushed researchers towards using rich procedural task generation systems controlled through complex continuous parameter spaces. In such complex task spaces, it is essential to rely on some form of Automatic Curriculum Learning (ACL) to adapt the task sampling distribution to a given learning agent, instead of randomly sampling tasks, as many could end up being either trivial or unfeasible. Since it is hard to get prior knowledge on such task spaces, many ACL algorithms explore the task space to detect progress niches over time, a costly tabula-rasa process that needs to be performed for each new learning agents, although they might have similarities in their capabilities profiles. To address this limitation, we introduce the concept of Meta-ACL, and formalize it in the context of black-box RL learners, i.e. algorithms seeking to generalize curriculum generation to an (unknown) distribution of learners. In this work, we present AGAIN, a first instantiation of Meta-ACL, and showcase its benefits for curriculum generation over classical ACL in multiple simulated environments including procedurally generated parkour environments with learners of varying morphologies. Videos and code are available at https://sites.google.com/view/meta-acl .

READ FULL TEXT

page 3

page 19

page 21

04/07/2020

Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning

A major challenge in the Deep RL (DRL) community is to train agents able...
03/31/2022

Robust Meta-Reinforcement Learning with Curriculum-Based Task Sampling

Meta-reinforcement learning (meta-RL) acquires meta-policies that show g...
10/16/2019

Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

We consider the problem of how a teacher algorithm can enable an unknown...
02/19/2020

Curriculum in Gradient-Based Meta-Reinforcement Learning

Gradient-based meta-learners such as Model-Agnostic Meta-Learning (MAML)...
07/01/2020

Adaptive Procedural Task Generation for Hard-Exploration Problems

We introduce Adaptive Procedural Task Generation (APT-Gen), an approach ...
11/07/2022

Curriculum-based Asymmetric Multi-task Reinforcement Learning

We introduce CAMRL, the first curriculum-based asymmetric multi-task lea...
09/27/2019

Automated curricula through setter-solver interactions

Reinforcement learning algorithms use correlations between policies and ...