Keep it Unreal: Bridging the Realism Gap for 2.5D Recognition with Geometry Priors Only

by   Sergey Zakharov, et al.

With the increasing availability of large databases of 3D CAD models, depth-based recognition methods can be trained on an uncountable number of synthetically rendered images. However, discrepancies with the real data acquired from various depth sensors still noticeably impede progress. Previous works adopted unsupervised approaches to generate more realistic depth data, but they all require real scans for training, even if unlabeled. This still represents a strong requirement, especially when considering real-life/industrial settings where real training images are hard or impossible to acquire, but texture-less 3D models are available. We thus propose a novel approach leveraging only CAD models to bridge the realism gap. Purely trained on synthetic data, playing against an extensive augmentation pipeline in an unsupervised manner, our generative adversarial network learns to effectively segment depth images and recover the clean synthetic-looking depth information even from partial occlusions. As our solution is not only fully decoupled from the real domains but also from the task-specific analytics, the pre-processed scans can be handed to any kind and number of recognition methods also trained on synthetic data. Through various experiments, we demonstrate how this simplifies their training and consistently enhances their performance, with results on par with the same methods trained on real data, and better than usual approaches doing the reverse mapping.



page 6

page 10

page 11

page 12

page 14

page 16

page 17

page 18


DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition

Recent progress in computer vision has been dominated by deep neural net...

Seeing Beyond Appearance - Mapping Real Images into Geometrical Domains for Unsupervised CAD-based Recognition

While convolutional neural networks are dominating the field of computer...

From Real to Synthetic and Back: Synthesizing Training Data for Multi-Person Scene Understanding

We present a method for synthesizing naturally looking images of multipl...

Generative adversarial network with object detector discriminator for enhanced defect detection on ultrasonic B-scans

Non-destructive testing is a set of techniques for defect detection in m...

Learning from Simulated and Unsupervised Images through Adversarial Training

With recent progress in graphics, it has become more tractable to train ...

Fake It Till You Make It: Face analysis in the wild using synthetic data alone

We demonstrate that it is possible to perform face-related computer visi...

A deep representation for depth images from synthetic data

Convolutional Neural Networks (CNNs) trained on large scale RGB database...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.