Asymmetric Actor Critic for Image-Based Robot Learning

10/18/2017
by   Lerrel Pinto, et al.
0

Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains. However, Robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator. While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator. In this work, we exploit the full state observability in the simulator to train better policies which take as input only partial observations (RGBD images). We do this by employing an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) gets rendered images as input. We show experimentally on a range of simulated tasks that using these asymmetric inputs significantly improves performance. Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block. We achieve this simulation to real world transfer without training on any real world data.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

research
12/13/2018

Soft Actor-Critic Algorithms and Applications

Model-free deep reinforcement learning (RL) algorithms have been success...
research
08/03/2022

AACC: Asymmetric Actor-Critic in Contextual Reinforcement Learning

Reinforcement Learning (RL) techniques have drawn great attention in man...
research
09/13/2023

Curriculum-based Sensing Reduction in Simulation to Real-World Transfer for In-hand Manipulation

Simulation to Real-World Transfer allows affordable and fast training of...
research
07/27/2020

Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation

Reinforcement Learning (RL), among other learning-based methods, represe...
research
04/30/2020

Sim-to-Real Transfer with Incremental Environment Complexity for Reinforcement Learning of Depth-Based Robot Navigation

Transferring learning-based models to the real world remains one of the ...
research
09/16/2022

Look where you look! Saliency-guided Q-networks for visual RL tasks

Deep reinforcement learning policies, despite their outstanding efficien...
research
11/19/2019

Attention Privileged Reinforcement Learning For Domain Transfer

Applying reinforcement learning (RL) to physical systems presents notabl...

Please sign up or login with your details

Forgot password? Click here to reset