Cross-View Panorama Image Synthesis

03/22/2022
by   Songsong Wu, et al.
4

In this paper, we tackle the problem of synthesizing a ground-view panorama image conditioned on a top-view aerial image, which is a challenging problem due to the large gap between the two image domains with different view-points. Instead of learning cross-view mapping in a feedforward pass, we propose a novel adversarial feedback GAN framework named PanoGAN with two key components: an adversarial feedback module and a dual branch discrimination strategy. First, the aerial image is fed into the generator to produce a target panorama image and its associated segmentation map in favor of model training with layout semantics. Second, the feature responses of the discriminator encoded by our adversarial feedback module are fed back to the generator to refine the intermediate representations, so that the generation performance is continually improved through an iterative generation process. Third, to pursue high-fidelity and semantic consistency of the generated panorama image, we propose a pixel-segmentation alignment mechanism under the dual branch discrimiantion strategy to facilitate cooperation between the generator and the discriminator. Extensive experimental results on two challenging cross-view image datasets show that PanoGAN enables high-quality panorama image generation with more convincing details than state-of-the-art approaches. The source code and trained models are available at <https://github.com/sswuai/PanoGAN>.

READ FULL TEXT

page 2

page 3

page 7

page 8

page 10

page 11

page 12

page 14

research
03/09/2018

Cross-View Image Synthesis using Conditional GANs

Learning to generate natural scenes has always been a challenging task i...
research
01/30/2023

GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis

Synthesizing high-fidelity complex images from text is challenging. Base...
research
04/15/2019

Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation

Cross-view image translation is challenging because it involves images w...
research
07/27/2022

Generator Knows What Discriminator Should Learn in Unconditional GANs

Recent methods for conditional image generation benefit from dense super...
research
10/19/2021

Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation

It is hard to generate an image at target view well for previous cross-v...
research
07/21/2020

Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder

Novel view synthesis often needs the paired data from both the source an...
research
12/09/2021

A Shared Representation for Photorealistic Driving Simulators

A powerful simulator highly decreases the need for real-world tests when...

Please sign up or login with your details

Forgot password? Click here to reset