Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation

04/21/2020
by   Ryan Julian, et al.
29

One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments. Despite this potential, most of the robot learning systems today are deployed as a fixed policy and they are not being adapted after their deployment. Can we efficiently adapt previously learned behaviors to new environments, objects and percepts in the real world? In this paper, we present a method and empirical evidence towards a robot learning framework that facilitates continuous adaption. In particular, we demonstrate how to adapt vision-based robotic manipulation policies to new variations by fine-tuning via off-policy reinforcement learning, including changes in background, object shape and appearance, lighting conditions, and robot morphology. Further, this adaptation uses less than 0.2 scratch. We find that our approach of adapting pre-trained policies leads to substantial performance gains over the course of fine-tuning, and that pre-training via RL is essential: training from scratch or adapting from supervised ImageNet features are both unsuccessful with such small amounts of data. We also find that these positive results hold in a limited continual learning setting, in which we repeatedly fine-tune a single lineage of policies using data from a succession of new tasks. Our empirical conclusions are consistently supported by experiments on simulated manipulation tasks, and by 52 unique fine-tuning experiments on a real robotic grasping system pre-trained on 580,000 grasps.

READ FULL TEXT

page 1

page 3

page 4

page 6

page 8

research
10/24/2019

RoboNet: Large-Scale Multi-Robot Learning

Robot learning has emerged as a promising tool for taming the complexity...
research
07/01/2021

Learning to See before Learning to Act: Visual Pre-training for Manipulation

Does having visual priors (e.g. the ability to detect objects) facilitat...
research
06/26/2023

Learning to Modulate pre-trained Models in RL

Reinforcement Learning (RL) has been successful in various domains like ...
research
06/05/2023

Continual Learning with Pretrained Backbones by Tuning in the Input Space

The intrinsic difficulty in adapting deep learning models to non-station...
research
01/17/2023

The SwaNNFlight System: On-the-Fly Sim-to-Real Adaptation via Anchored Learning

Reinforcement Learning (RL) agents trained in simulated environments and...
research
11/12/2018

Adaptive Target Recognition: A Case Study Involving Airport Baggage Screening

This work addresses the question whether it is possible to design a comp...
research
10/11/2021

Learning a subspace of policies for online adaptation in Reinforcement Learning

Deep Reinforcement Learning (RL) is mainly studied in a setting where th...

Please sign up or login with your details

Forgot password? Click here to reset