Cross-View Image Synthesis using Conditional GANs

03/09/2018
by   Krishna Regmi, et al.
0

Learning to generate natural scenes has always been a challenging task in computer vision. It is even more painstaking when the generation is conditioned on images with drastically different views. This is mainly because understanding, corresponding, and transforming appearance and semantic information across views is not trivial. In this paper, we attempt to solve the novel problem of cross-view image synthesis, aerial to street view and vice versa, using conditional generative adversarial networks (cGAN). Two new architectures called Crossview Fork (XFork) and Crossview Sequential (X-Seq) are proposed to generate scenes with resolutions of 64x64 and 256x256 pixels. X-Fork architecture has a single discriminator and a single generator. The generator hallucinates both the image and its semantic segmentation in the target view. X-Seq architecture utilizes two cGANs. The first one generates the target image which is subsequently fed to the second cGAN for generating its corresponding semantic segmentation map. The feedback from the second cGAN helps the first cGAN generate sharper images. Both of our proposed architectures learn to generate natural images as well as their semantic segmentation maps. Extensive qualitative and quantitative evaluations support the effectiveness of our frameworks, compared to two state of the art methods, for natural scene generation across drastically different views.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 8

research
08/14/2018

Cross-view image synthesis using geometry-guided conditional GANs

We address the problem of generating images across two drastically diffe...
research
07/20/2020

Cross-View Image Synthesis with Deformable Convolution and Attention Mechanism

Learning to generate natural scenes has always been a daunting task in c...
research
03/22/2022

Cross-View Panorama Image Synthesis

In this paper, we tackle the problem of synthesizing a ground-view panor...
research
10/19/2017

Be Your Own Prada: Fashion Synthesis with Structural Coherence

We present a novel and effective approach for generating new clothing on...
research
04/07/2017

Learning Where to Look: Data-Driven Viewpoint Set Selection for 3D Scenes

The use of rendered images, whether from completely synthetic datasets o...
research
08/20/2020

Generative View Synthesis: From Single-view Semantics to Novel-view Images

Content creation, central to applications such as virtual reality, can b...
research
12/08/2021

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation

Editing facial expressions by only changing what we want is a long-stand...

Please sign up or login with your details

Forgot password? Click here to reset