Lifelong Robotic Reinforcement Learning by Retaining Experiences

09/19/2021
by   Annie Xie, et al.
12

Multi-task learning ideally allows robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In reality, the tasks that the robot learns arrive sequentially, depending on the user and the robot's current environment. In this work, we study a practical sequential multi-task RL problem that is motivated by the practical constraints of physical robotic systems, and derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding impractical round-robin data collection. On a Franka Emika Panda robot arm, our approach incrementally learns ten challenging tasks, including bottle capping and block insertion.

READ FULL TEXT

page 1

page 2

page 5

research
04/16/2021

MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

General-purpose robotic systems must master a large repertoire of divers...
research
07/11/2022

Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning

Reinforcement learning (RL) algorithms hold the promise of enabling auto...
research
05/10/2021

Efficient Self-Supervised Data Collection for Offline Robot Learning

A practical approach to robot reinforcement learning is to first collect...
research
11/12/2020

Reinforcement Learning with Videos: Combining Offline Observations with Interaction

Reinforcement learning is a powerful framework for robots to acquire ski...
research
02/19/2023

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

This work aims to push the limits of agility for bipedal robots by enabl...
research
07/26/2023

Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition

We present a framework for robot skill acquisition, which 1) efficiently...
research
02/20/2022

Towards a Framework for Comparing the Complexity of Robotic Tasks

We are motivated by the problem of comparing the complexity of one robot...

Please sign up or login with your details

Forgot password? Click here to reset