A User's Guide to Calibrating Robotics Simulators

by   Bhairav Mehta, et al.

Simulators are a critical component of modern robotics research. Strategies for both perception and decision making can be studied in simulation first before deployed to real world systems, saving on time and costs. Despite significant progress on the development of sim-to-real algorithms, the analysis of different methods is still conducted in an ad-hoc manner, without a consistent set of tests and metrics for comparison. This paper fills this gap and proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms. We open-source the benchmark, training data, and trained models, which can be found at https://github.com/NVlabs/sim-parameter-estimation.



page 2

page 6


robo-gym – An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robots

Applying Deep Reinforcement Learning (DRL) to complex tasks in the field...

Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience

We consider the problem of transferring policies to the real world by tr...

Benchmarking Simulated Robotic Manipulation through a Real World Dataset

We present a benchmark to facilitate simulated manipulation; an attempt ...

Simulation and Control of Deformable Autonomous Airships in Turbulent Wind

Abstract. Fixed wing and multirotor UAVs are common in the field of robo...

Quad2Plane: An Intermediate Training Procedure for Online Exploration in Aerial Robotics via Receding Horizon Control

Data driven robotics relies upon accurate real-world representations to ...

Benchmarks for Deep Off-Policy Evaluation

Off-policy evaluation (OPE) holds the promise of being able to leverage ...

S2AND: A Benchmark and Evaluation System for Author Name Disambiguation

Author Name Disambiguation (AND) is the task of resolving which author m...

Code Repositories


The code accompaniment for the CoRL 2020 paper: A User's Guide to Calibrating Robotics Simulators (https://arxiv.org/abs/2011.08985), from NVIDIA Research Seattle.

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.