Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

by   Rémy Portelas, et al.

We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters controlling a stochastic procedural generation of environments. Because it does not initially know the capacities of its student, a key challenge for the teacher is to discover which environments are easy, difficult or unlearnable, and in what order to propose them to maximize the efficiency of learning over the learnable ones. To achieve this, this problem is transformed into a surrogate continuous bandit problem where the teacher samples environments in order to maximize absolute learning progress of its student. We present a new algorithm modeling absolute learning progress with Gaussian mixture models (ALP-GMM). We also adapt existing algorithms and provide a complete study in the context of DRL. Using parameterized variants of the BipedalWalker environment, we study their efficiency to personalize a learning curriculum for different learners (embodiments), their robustness to the ratio of learnable/unlearnable environments, and their scalability to non-linear and high-dimensional parameter spaces. Videos and code are available at https://github.com/flowersteam/teachDeepRL.


page 6

page 12

page 14

page 15

page 18

page 19


TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL

Training autonomous agents able to generalize to multiple tasks is a key...

Meta Automatic Curriculum Learning

A major challenge in the Deep RL (DRL) community is to train agents able...

Mastering Rate based Curriculum Learning

Recent automatic curriculum learning algorithms, and in particular Teach...

Teacher-Student Curriculum Learning

We propose Teacher-Student Curriculum Learning (TSCL), a framework for a...

DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning

Deep reinforcement learning (RL) has shown great empirical successes, bu...

Autonomous Exploration and Mapping for Mobile Robots via Cumulative Curriculum Reinforcement Learning

Deep reinforcement learning (DRL) has been widely applied in autonomous ...

Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning

A major challenge in the Deep RL (DRL) community is to train agents able...

Code Repositories


Thesis on proc generation of environments using ppo as a student on MiniGrid environments

view repo

Please sign up or login with your details

Forgot password? Click here to reset