Autoencoder for Synthetic to Real Generalization: From Simple to More Complex Scenes

04/01/2022
by   Steve Dias Da Cruz, et al.
33

Learning on synthetic data and transferring the resulting properties to their real counterparts is an important challenge for reducing costs and increasing safety in machine learning. In this work, we focus on autoencoder architectures and aim at learning latent space representations that are invariant to inductive biases caused by the domain shift between simulated and real images showing the same scenario. We train on synthetic images only, present approaches to increase generalizability and improve the preservation of the semantics to real datasets of increasing visual complexity. We show that pre-trained feature extractors (e.g. VGG) can be sufficient for generalization on images of lower complexity, but additional improvements are required for visually more complex scenes. To this end, we demonstrate a new sampling technique, which matches semantically important parts of the image, while randomizing the other parts, leads to salient feature extraction and a neglection of unimportant parts. This helps the generalization to real data and we further show that our approach outperforms fine-tuned classification models.

READ FULL TEXT

page 3

page 4

page 5

research
03/11/2015

Learning Classifiers from Synthetic Data Using a Multichannel Autoencoder

We propose a method for using synthetic data to help learning classifier...
research
04/06/2021

Contrastive Syn-to-Real Generalization

Training on synthetic data can be beneficial for label or data-scarce sc...
research
05/19/2020

Focus on defocus: bridging the synthetic to real domain gap for depth estimation

Data-driven depth estimation methods struggle with the generalization ou...
research
10/29/2017

On Pre-Trained Image Features and Synthetic Images for Deep Learning

Deep Learning methods usually require huge amounts of training data to p...
research
05/16/2022

Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencoder

Domain generalization aims to improve the generalization capability of m...
research
04/26/2019

DeepFreak: Learning Crystallography Diffraction Patterns with Automated Machine Learning

Serial crystallography is the field of science that studies the structur...
research
01/06/2022

ITSA: An Information-Theoretic Approach to Automatic Shortcut Avoidance and Domain Generalization in Stereo Matching Networks

State-of-the-art stereo matching networks trained only on synthetic data...

Please sign up or login with your details

Forgot password? Click here to reset