Towards Realistic 3D Embedding via View Alignment

07/14/2020
by   Fangneng Zhan, et al.
0

Recent advances in generative adversarial networks (GANs) have achieved great success in automated image composition that generates new images by embedding interested foreground objects into background images automatically. On the other hand, most existing works deal with foreground objects in two-dimensional (2D) images though foreground objects in three-dimensional (3D) models are more flexible with 360-degree view freedom. This paper presents an innovative View Alignment GAN (VA-GAN) that composes new images by embedding 3D models into 2D background images realistically and automatically. VA-GAN consists of a texture generator and a differential discriminator that are inter-connected and end-to-end trainable. The differential discriminator guides to learn geometric transformation from background images so that the composed 3D models can be aligned with the background images with realistic poses and views. The texture generator adopts a novel view encoding mechanism for generating accurate object textures for the 3D models under the estimated views. Extensive experiments over two synthesis tasks (car synthesis with KITTI and pedestrian synthesis with Cityscapes) show that VA-GAN achieves high-fidelity composition qualitatively and quantitatively as compared with state-of-the-art generation methods.

READ FULL TEXT

page 2

page 4

page 5

page 6

page 8

research
12/14/2018

Spatial Fusion GAN for Image Synthesis

Recent advances in generative adversarial networks (GANs) have shown gre...
research
07/07/2021

FBC-GAN: Diverse and Flexible Image Synthesis via Foreground-Background Composition

Generative Adversarial Networks (GANs) have become the de-facto standard...
research
09/17/2020

Adversarial Image Composition with Auxiliary Illumination

Dealing with the inconsistency between a foreground object and a backgro...
research
05/12/2019

Adaptive Composition GAN towards Realistic Image Synthesis

Despite the rapid progress of generative adversarial networks (GANs) in ...
research
03/28/2021

Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection

Automated defect inspection is critical for effective and efficient main...
research
01/22/2023

BallGAN: 3D-aware Image Synthesis with a Spherical Background

3D-aware GANs aim to synthesize realistic 3D scenes such that they can b...
research
01/19/2019

Synthesizing facial photometries and corresponding geometries using generative adversarial networks

Artificial data synthesis is currently a well studied topic with useful ...

Please sign up or login with your details

Forgot password? Click here to reset