Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

09/21/2021
by   Bohan Wu, et al.
5

In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks. Reinforcement learning (RL) is a promising approach for acquiring short-horizon skills autonomously. However, the focus of RL algorithms has largely been on the success of those individual skills, more so than learning and grounding a large repertoire of skills that can be sequenced to complete extended multi-stage tasks. The latter demands robustness and persistence, as errors in skills can compound over time, and may require the robot to have a number of primitive skills in its repertoire, rather than just one. To this end, we introduce EMBR, a model-based RL method for learning primitive skills that are suitable for completing long-horizon visuomotor tasks. EMBR learns and plans using a learned model, critic, and success classifier, where the success classifier serves both as a reward function for RL and as a grounding mechanism to continuously detect if the robot should retry a skill when unsuccessful or under perturbations. Further, the learned model is task-agnostic and trained using data from all skills, enabling the robot to efficiently learn a number of distinct primitives. These visuomotor primitive skills and their associated pre- and post-conditions can then be directly combined with off-the-shelf symbolic planners to complete long-horizon tasks. On a Franka Emika robot arm, we find that EMBR enables the robot to complete three long-horizon visuomotor tasks at 85 organizing an office desk, a file cabinet, and drawers, which require sequencing up to 12 skills, involve 14 unique learned primitives, and demand generalization to novel objects.

READ FULL TEXT

page 1

page 6

page 8

research
05/23/2023

M-EMBER: Tackling Long-Horizon Mobile Manipulation via Factorized Domain Transfer

In this paper, we propose a method to create visuomotor mobile manipulat...
research
07/11/2022

Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning

Problems which require both long-horizon planning and continuous control...
research
11/04/2022

Mixline: A Hybrid Reinforcement Learning Framework for Long-horizon Bimanual Coffee Stirring Task

Bimanual activities like coffee stirring, which require coordination of ...
research
04/25/2022

Skill-based Meta-Reinforcement Learning

While deep reinforcement learning methods have shown impressive results ...
research
12/08/2021

Specializing Versatile Skill Libraries using Local Mixture of Experts

A long-cherished vision in robotics is to equip robots with skills that ...
research
06/07/2022

Meta-Learning Transferable Parameterized Skills

We propose a novel parameterized skill-learning algorithm that aims to l...
research
07/31/2023

Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot

Reinforcement learning is still struggling with solving long-horizon sur...

Please sign up or login with your details

Forgot password? Click here to reset