Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap

06/02/2020
by   Jacky Liang, et al.
0

Training robotic policies in simulation suffers from the sim-to-real gap, as simulated dynamics can be different from real-world dynamics. Past works tackled this problem through domain randomization and online system-identification. The former is sensitive to the manually-specified training distribution of dynamics parameters and can result in behaviors that are overly conservative. The latter requires learning policies that concurrently perform the task and generate useful trajectories for system identification. In this work, we propose and analyze a framework for learning exploration policies that explicitly perform task-oriented exploration actions to identify task-relevant system parameters. These parameters are then used by model-based trajectory optimization algorithms to perform the task in the real world. We instantiate the framework in simulation with the Linear Quadratic Regulator as well as in the real world with pouring and object dragging tasks. Experiments show that task-oriented exploration helps model-based policies adapt to systems with initially unknown parameters, and it leads to better task performance than task-agnostic exploration.

READ FULL TEXT

page 1

page 4

page 7

page 8

research
02/09/2023

AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer

Simulation parameter settings such as contact models and object geometry...
research
10/12/2018

Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience

We consider the problem of transferring policies to the real world by tr...
research
03/08/2023

Learning Exploration Strategies to Solve Real-World Marble Runs

Tasks involving locally unstable or discontinuous dynamics (such as bifu...
research
04/14/2021

Small World Model for scaling up prediction result based on SEIR model

Data-driven epidemic simulation helps better policymaking. Compared with...
research
10/18/2017

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

Simulations are attractive environments for training agents as they prov...
research
04/15/2021

Auto-Tuned Sim-to-Real Transfer

Policies trained in simulation often fail when transferred to the real w...
research
02/06/2020

Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation

This paper addresses a new strategy called Simulation-to-Real-to-Simulat...

Please sign up or login with your details

Forgot password? Click here to reset