Learning Spatial Pyramid Attentive Pooling in Image Synthesis and Image-to-Image Translation

01/18/2019
by   Wei Sun, et al.
8

Image synthesis and image-to-image translation are two important generative learning tasks. Remarkable progress has been made by learning Generative Adversarial Networks (GANs) goodfellow2014generative and cycle-consistent GANs (CycleGANs) zhu2017unpaired respectively. This paper presents a method of learning Spatial Pyramid Attentive Pooling (SPAP) which is a novel architectural unit and can be easily integrated into both generators and discriminators in GANs and CycleGANs. The proposed SPAP integrates Atrous spatial pyramid chen2018deeplab, a proposed cascade attention mechanism and residual connections he2016deep. It leverages the advantages of the three components to facilitate effective end-to-end generative learning: (i) the capability of fusing multi-scale information by ASPP; (ii) the capability of capturing relative importance between both spatial locations (especially multi-scale context) or feature channels by attention; (iii) the capability of preserving information and enhancing optimization feasibility by residual connections. Coarse-to-fine and fine-to-coarse SPAP are studied and intriguing attention maps are observed in both tasks. In experiments, the proposed SPAP is tested in GANs on the Celeba-HQ-128 dataset karras2017progressive, and tested in CycleGANs on the Image-to-Image translation datasets including the Cityscape dataset cordts2016cityscapes, Facade and Aerial Maps dataset zhu2017unpaired, both obtaining better performance.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
01/24/2019

Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation

Current approaches have made great progress on image-to-image translatio...
research
02/12/2021

Broad-UNet: Multi-scale feature learning for nowcasting tasks

Weather nowcasting consists of predicting meteorological components in t...
research
02/03/2020

Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation

We propose a novel model named Multi-Channel Attention Selection Generat...
research
02/28/2019

Two-phase Hair Image Synthesis by Self-Enhancing Generative Model

Generating plausible hair image given limited guidance, such as sparse s...
research
07/25/2019

Co-Evolutionary Compression for Unpaired Image Translation

Generative adversarial networks (GANs) have been successfully used for c...
research
08/17/2021

Transferring Knowledge with Attention Distillation for Multi-Domain Image-to-Image Translation

Gradient-based attention modeling has been used widely as a way to visua...
research
12/09/2020

Positional Encoding as Spatial Inductive Bias in GANs

SinGAN shows impressive capability in learning internal patch distributi...

Please sign up or login with your details

Forgot password? Click here to reset