Persistent Reinforcement Learning via Subgoal Curricula

07/27/2021
by   Archit Sharma, et al.
2

Reinforcement learning (RL) promises to enable autonomous acquisition of complex behaviors for diverse agents. However, the success of current reinforcement learning algorithms is predicated on an often under-emphasised requirement – each trial needs to start from a fixed initial state distribution. Unfortunately, resetting the environment to its initial state after each trial requires substantial amount of human supervision and extensive instrumentation of the environment which defeats the purpose of autonomous reinforcement learning. In this work, we propose Value-accelerated Persistent Reinforcement Learning (VaPRL), which generates a curriculum of initial states such that the agent can bootstrap on the success of easier tasks to efficiently learn harder tasks. The agent also learns to reach the initial states proposed by the curriculum, minimizing the reliance on human interventions into the learning. We observe that VaPRL reduces the interventions required by three orders of magnitude compared to episodic RL while outperforming prior state-of-the art methods for reset-free RL both in terms of sample efficiency and asymptotic performance on a variety of simulated robotics problems.

READ FULL TEXT
research
12/17/2021

Autonomous Reinforcement Learning: Formalism and Benchmarking

Reinforcement learning (RL) provides a naturalistic framing for learning...
research
05/17/2023

Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum

While reinforcement learning (RL) has achieved great success in acquirin...
research
03/10/2020

Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

Reinforcement learning (RL) is a popular paradigm for addressing sequent...
research
08/04/2021

Parallelized Reverse Curriculum Generation

For reinforcement learning (RL), it is challenging for an agent to maste...
research
02/17/2022

Robust Reinforcement Learning via Genetic Curriculum

Achieving robust performance is crucial when applying deep reinforcement...
research
04/05/2022

Automating Reinforcement Learning with Example-based Resets

Deep reinforcement learning has enabled robots to learn motor skills fro...
research
10/17/2022

You Only Live Once: Single-Life Reinforcement Learning

Reinforcement learning algorithms are typically designed to learn a perf...

Please sign up or login with your details

Forgot password? Click here to reset