Knowing the Distance: Understanding the Gap Between Synthetic and Real Data For Face Parsing

03/27/2023
by   Eli Friedman, et al.
0

The use of synthetic data for training computer vision algorithms has become increasingly popular due to its cost-effectiveness, scalability, and ability to provide accurate multi-modality labels. Although recent studies have demonstrated impressive results when training networks solely on synthetic data, there remains a performance gap between synthetic and real data that is commonly attributed to lack of photorealism. The aim of this study is to investigate the gap in greater detail for the face parsing task. We differentiate between three types of gaps: distribution gap, label gap, and photorealism gap. Our findings show that the distribution gap is the largest contributor to the performance gap, accounting for over 50 addressing this gap and accounting for the labels gap, we demonstrate that a model trained on synthetic data achieves comparable results to one trained on a similar amount of real data. This suggests that synthetic data is a viable alternative to real data, especially when real data is limited or difficult to obtain. Our study highlights the importance of content diversity in synthetic datasets and challenges the notion that the photorealism gap is the most critical factor affecting the performance of computer vision models trained on synthetic data.

READ FULL TEXT

page 2

page 5

page 10

research
09/30/2021

Fake It Till You Make It: Face analysis in the wild using synthetic data alone

We demonstrate that it is possible to perform face-related computer visi...
research
10/19/2018

Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing

We introduce Synscapes -- a synthetic dataset for street scene parsing c...
research
09/17/2015

Learning from Synthetic Data Using a Stacked Multichannel Autoencoder

Learning from synthetic data has many important and practical applicatio...
research
09/27/2020

STAN: Synthetic Network Traffic Generation using Autoregressive Neural Models

Deep learning models have achieved great success in recent years. Howeve...
research
07/27/2018

Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics

Widely used in news, business, and educational media, infographics are h...
research
11/30/2020

Sim2SG: Sim-to-Real Scene Graph Generation for Transfer Learning

Scene graph (SG) generation has been gaining a lot of traction recently....
research
09/11/2021

MLReal: Bridging the gap between training on synthetic data and real data applications in machine learning

Among the biggest challenges we face in utilizing neural networks traine...

Please sign up or login with your details

Forgot password? Click here to reset