SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning

01/15/2021
by   Yifeng Jiang, et al.
0

As learning-based approaches progress towards automating robot controllers design, transferring learned policies to new domains with different dynamics (e.g. sim-to-real transfer) still demands manual effort. This paper introduces SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics simulator to match the simulated trajectories to the ones from the target domain, using a learned discriminative loss to address the limitations associated with manual loss design. Our hybrid simulator combines neural networks and traditional physics simulaton to balance expressiveness and generalizability, and alleviates the need for a carefully selected parameter set in System ID. Once the hybrid simulator is identified via adversarial reinforcement learning, it can be used to refine policies for the target domain, without the need to collect more data. We show that our approach outperforms multiple strong baselines on six robotic locomotion tasks for domain adaptation.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/19/2018

Learning Sampling Policies for Domain Adaptation

We address the problem of semi-supervised domain adaptation of classific...
10/05/2016

EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

Sample complexity and safety are major challenges when learning policies...
05/10/2019

Domain Adversarial Reinforcement Learning for Partial Domain Adaptation

Partial domain adaptation aims to transfer knowledge from a label-rich s...
03/12/2020

Fisher Deep Domain Adaptation

Deep domain adaptation models learn a neural network in an unlabeled tar...
09/08/2019

Compound Domain Adaptation in an Open World

Existing works on domain adaptation often assume clear boundaries betwee...
08/04/2020

Stochastic Grounded Action Transformation for Robot Learning in Simulation

Robot control policies learned in simulation do not often transfer well ...
09/26/2022

Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization

Training deep reinforcement learning (DRL) locomotion policies often req...