Stochastic Grounded Action Transformation for Robot Learning in Simulation

by   Siddharth Desai, et al.

Robot control policies learned in simulation do not often transfer well to the real world. Many existing solutions to this sim-to-real problem, such as the Grounded Action Transformation (GAT) algorithm, seek to correct for or ground these differences by matching the simulator to the real world. However, the efficacy of these approaches is limited if they do not explicitly account for stochasticity in the target environment. In this work, we analyze the problems associated with grounding a deterministic simulator in a stochastic real world environment, and we present examples where GAT fails to transfer a good policy due to stochastic transitions in the target domain. In response, we introduce the Stochastic Grounded Action Transformation(SGAT) algorithm,which models this stochasticity when grounding the simulator. We find experimentally for both simulated and physical target domains that SGAT can find policies that are robust to stochasticity in the target domain


page 2

page 5

page 6


Reinforced Grounded Action Transformation for Sim-to-Real Transfer

Robots can learn to do complex tasks in simulation, but often, learned b...

Off Environment Evaluation Using Convex Risk Minimization

Applying reinforcement learning (RL) methods on robots typically involve...

An Imitation from Observation Approach to Sim-to-Real Transfer

The sim to real transfer problem deals with leveraging large amounts of ...

Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding

The ability to transfer a policy from one environment to another is a pr...

SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning

As learning-based approaches progress towards automating robot controlle...

An Objective Assessment of the Utility of a Driving Simulator for Low Mu Testing

Driving simulators can be used to test vehicle designs earlier, prior to...

Close the Visual Domain Gap by Physics-Grounded Active Stereovision Depth Sensor Simulation

In this paper, we focus on the simulation of active stereovision depth s...