TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL

by   Clément Romac, et al.

Training autonomous agents able to generalize to multiple tasks is a key target of Deep Reinforcement Learning (DRL) research. In parallel to improving DRL algorithms themselves, Automatic Curriculum Learning (ACL) study how teacher algorithms can train DRL agents more efficiently by adapting task selection to their evolving abilities. While multiple standard benchmarks exist to compare DRL agents, there is currently no such thing for ACL algorithms. Thus, comparing existing approaches is difficult, as too many experimental parameters differ from paper to paper. In this work, we identify several key challenges faced by ACL algorithms. Based on these, we present TeachMyAgent (TA), a benchmark of current ACL algorithms leveraging procedural task generation. It includes 1) challenge-specific unit-tests using variants of a procedural Box2D bipedal walker environment, and 2) a new procedural Parkour environment combining most ACL challenges, making it ideal for global performance assessment. We then use TeachMyAgent to conduct a comparative study of representative existing approaches, showcasing the competitiveness of some ACL algorithms that do not use expert knowledge. We also show that the Parkour environment remains an open problem. We open-source our environments, all studied ACL algorithms (collected from open-source code or re-implemented), and DRL students in a Python package available at https://github.com/flowersteam/TeachMyAgent.


page 19

page 26

page 29

page 32

page 33

page 35

page 37

page 38


ChainerRL: A Deep Reinforcement Learning Library

In this paper, we introduce ChainerRL, an open-source Deep Reinforcement...

Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

We consider the problem of how a teacher algorithm can enable an unknown...

Autonomous Exploration and Mapping for Mobile Robots via Cumulative Curriculum Reinforcement Learning

Deep reinforcement learning (DRL) has been widely applied in autonomous ...

Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning

A major challenge in the Deep RL (DRL) community is to train agents able...

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

Building embodied autonomous agents capable of participating in social i...

Automatic Curriculum Learning For Deep RL: A Short Survey

Automatic Curriculum Learning (ACL) has become a cornerstone of recent s...

Meta Automatic Curriculum Learning

A major challenge in the Deep RL (DRL) community is to train agents able...

Please sign up or login with your details

Forgot password? Click here to reset