Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics

09/13/2018
by   Jeroen van Baar, et al.
0

Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved, is that the amount of time a robot is occupied learning a task---rather than being productive---can be reduced by transferring the learned task to the real robot. Transfer learning requires some amount of fine-tuning on the real robot. For tasks which involve complex (non-linear) dynamics, the fine-tuning itself may take a substantial amount of time. In order to reduce the amount of fine-tuning we propose to learn robustified controllers in simulation. Robustified controllers are learned by exploiting the ability to change simulation parameters (both appearance and dynamics) for successive training episodes. An additional benefit for this approach is that it alleviates the precise determination of physics parameters for the simulator, which is a non-trivial task. We demonstrate our proposed approach on a real setup in which a robot aims to solve a maze puzzle, which involves complex dynamics due to static friction and potentially large accelerations. We show that the amount of fine-tuning in transfer learning for a robustified controller is substantially reduced compared to a non-robustified controller.

READ FULL TEXT
research
02/10/2021

Transfer Reinforcement Learning across Homotopy Classes

The ability for robots to transfer their learned knowledge to new tasks ...
research
07/25/2017

Mutual Alignment Transfer Learning

Training robots for operation in the real world is a complex, time consu...
research
08/06/2021

Attainment Regions in Feature-Parameter Space for High-Level Debugging in Autonomous Robots

Understanding a controller's performance in different scenarios is cruci...
research
05/29/2020

Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera

Even though the peg-hole insertion is one of the well-studied problems i...
research
05/02/2019

From Video Game to Real Robot: The Transfer between Action Spaces

Training agents with reinforcement learning based techniques requires th...
research
07/19/2021

Know Thyself: Transferable Visuomotor Control Through Robot-Awareness

Training visuomotor robot controllers from scratch on a new robot typica...
research
03/11/2018

Learning Partially Structured Environmental Dynamics for Marine Robotic Navigation

We investigate the scenario that a robot needs to reach a designated goa...

Please sign up or login with your details

Forgot password? Click here to reset