Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

11/13/2020
by   Bryan Chen, et al.
0

Vision-based robotics often separates the control loop into one module for perception and a separate module for control. It is possible to train the whole system end-to-end (e.g. with deep RL), but doing it "from scratch" comes with a high sample complexity cost and the final result is often brittle, failing unexpectedly if the test environment differs from that of training. We study the effects of using mid-level visual representations (features learned asynchronously for traditional computer vision objectives), as a generic and easy-to-decode perceptual state in an end-to-end RL framework. Mid-level representations encode invariances about the world, and we show that they aid generalization, improve sample complexity, and lead to a higher final performance. Compared to other approaches for incorporating invariances, such as domain randomization, asynchronously trained mid-level representations scale better: both to harder problems and to larger domain shifts. In practice, this means that mid-level representations could be used to successfully train policies for tasks where domain randomization and learning-from-scratch failed. We report results on both manipulation and navigation tasks, and for navigation include zero-shot sim-to-real experiments on real robots.

READ FULL TEXT

page 3

page 4

page 5

research
12/31/2018

Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active Tasks

One of the ultimate promises of computer vision is to help robotic agent...
research
04/29/2021

Pre-training of Deep RL Agents for Improved Learning under Domain Randomization

Visual domain randomization in simulated environments is a widely used m...
research
02/12/2022

End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representation

We present an end-to-end Reinforcement Learning(RL) framework for roboti...
research
01/16/2018

Learning to Navigate: Exploiting Deep Networks to Inform Sample-Based Planning During Vision-Based Navigation

Recent applications of deep learning to navigation have generated end-to...
research
12/23/2019

Learning to Navigate Using Mid-Level Visual Priors

How much does having visual priors about the world (e.g. the fact that t...
research
02/20/2021

How To Train Your HERON

In this paper we apply Deep Reinforcement Learning (Deep RL) and Domain ...
research
03/10/2019

Affordance Learning for End-to-End Visuomotor Robot Control

Training end-to-end deep robot policies requires a lot of domain-, task-...

Please sign up or login with your details

Forgot password? Click here to reset