DeepAI AI Chat
Log In Sign Up

DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer

by   Ajay Kumar Tanwani, et al.
berkeley college

Generating large-scale synthetic data in simulation is a feasible alternative to collecting/labelling real data for training vision-based deep learning models, albeit the modelling inaccuracies do not generalize to the physical world. In this paper, we present a domain-invariant representation learning (DIRL) algorithm to adapt deep models to the physical environment with a small amount of real data. Existing approaches that only mitigate the covariate shift by aligning the marginal distributions across the domains and assume the conditional distributions to be domain-invariant can lead to ambiguous transfer in real scenarios. We propose to jointly align the marginal (input domains) and the conditional (output labels) distributions to mitigate the covariate and the conditional shift across the domains with adversarial learning, and combine it with a triplet distribution loss to make the conditional distributions disjoint in the shared feature space. Experiments on digit domains yield state-of-the-art performance on challenging benchmarks, while sim-to-real transfer of object recognition for vision-based decluttering with a mobile robot improves from 26.8 a wide variety of objects. Code and supplementary details are available at


page 13

page 14


Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment

Unsupervised knowledge transfer has a great potential to improve the gen...

Domain-Invariant Proposals based on a Balanced Domain Classifier for Object Detection

Object recognition from images means to automatically find object(s) of ...

Transfer Learning with Dynamic Adversarial Adaptation Network

The recent advances in deep transfer learning reveal that adversarial le...

Learning Domain-Invariant Relationship with Instrumental Variable for Domain Generalization

Domain generalization (DG) aims to learn from multiple source domains a ...

Conditional Bures Metric for Domain Adaptation

As a vital problem in classification-oriented transfer, unsupervised dom...

Generalized Label Shift Correction via Minimum Uncertainty Principle: Theory and Algorithm

As a fundamental problem in machine learning, dataset shift induces a pa...

Revisiting the Threat Space for Vision-based Keystroke Inference Attacks

A vision-based keystroke inference attack is a side-channel attack in wh...