Reinforced Grounded Action Transformation for Sim-to-Real Transfer

08/04/2020
by   Haresh Karnan, et al.
0

Robots can learn to do complex tasks in simulation, but often, learned behaviors fail to transfer well to the real world due to simulator imperfections (the reality gap). Some existing solutions to this sim-to-real problem, such as Grounded Action Transformation (GAT), use a small amount of real-world experience to minimize the reality gap by grounding the simulator. While very effective in certain scenarios, GAT is not robust on problems that use complex function approximation techniques to model a policy. In this paper, we introduce Reinforced Grounded Action Transformation(RGAT), a new sim-to-real technique that uses Reinforcement Learning (RL) not only to update the target policy in simulation, but also to perform the grounding step itself. This novel formulation allows for end-to-end training during the grounding step, which, compared to GAT, produces a better grounded simulator. Moreover, we show experimentally in several MuJoCo domains that our approach leads to successful transfer for policies modeled using neural networks.

READ FULL TEXT
research
08/04/2020

Stochastic Grounded Action Transformation for Robot Learning in Simulation

Robot control policies learned in simulation do not often transfer well ...
research
08/04/2020

An Imitation from Observation Approach to Sim-to-Real Transfer

The sim to real transfer problem deals with leveraging large amounts of ...
research
08/28/2023

LLM Powered Sim-to-real Transfer for Traffic Signal Control

Numerous solutions are proposed for the Traffic Signal Control (TSC) tas...
research
01/28/2022

Close the Visual Domain Gap by Physics-Grounded Active Stereovision Depth Sensor Simulation

In this paper, we focus on the simulation of active stereovision depth s...
research
03/03/2020

Traversing the Reality Gap via Simulator Tuning

The large demand for simulated data has made the reality gap a problem o...
research
02/23/2020

Deep Reinforcement Learning with Linear Quadratic Regulator Regions

Practitioners often rely on compute-intensive domain randomization to en...
research
07/10/2019

Assessing Transferability from Simulation to Reality for Reinforcement Learning

Learning robot control policies from physics simulations is of great int...

Please sign up or login with your details

Forgot password? Click here to reset