Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models

09/27/2020
by   Lei M. Zhang, et al.
13

We propose a method to predict the sim-to-real transfer performance of RL policies. Our transfer metric simplifies the selection of training setups (such as algorithm, hyperparameters, randomizations) and policies in simulation, without the need for extensive and time-consuming real-world rollouts. A probabilistic dynamics model is trained alongside the policy and evaluated on a fixed set of real-world trajectories to obtain the transfer metric. Experiments show that the transfer metric is highly correlated with policy performance in both simulated and real-world robotic environments for complex manipulation tasks. We further show that the transfer metric can predict the effect of training setups on policy transfer performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/12/2018

Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience

We consider the problem of transferring policies to the real world by tr...
research
08/23/2022

Generating people flow from architecture of real unseen environments

Mapping people dynamics is a crucial skill, because it enables robots to...
research
06/04/2019

Off-Policy Evaluation via Off-Policy Classification

In this work, we consider the problem of model selection for deep reinfo...
research
10/27/2022

Provable Sim-to-real Transfer in Continuous Domain with Partial Observations

Sim-to-real transfer trains RL agents in the simulated environments and ...
research
07/02/2022

Learning Switching Criteria for Sim2Real Transfer of Robotic Fabric Manipulation Policies

Simulation-to-reality transfer has emerged as a popular and highly succe...
research
06/06/2023

A metric for assessing and optimizing data-driven prognostic algorithms for predictive maintenance

Prognostic Health Management aims to predict the Remaining Useful Life (...
research
10/11/2016

Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model

Developing control policies in simulation is often more practical and sa...

Please sign up or login with your details

Forgot password? Click here to reset