Multimodal Image-to-Image Translation via a Single Generative Adversarial Network

08/04/2020
by   Shihua Huang, et al.
0

Despite significant advances in image-to-image (I2I) translation with Generative Adversarial Networks (GANs) have been made, it remains challenging to effectively translate an image to a set of diverse images in multiple target domains using a pair of generator and discriminator. Existing multimodal I2I translation methods adopt multiple domain-specific content encoders for different domains, where each domain-specific content encoder is trained with images from the same domain only. Nevertheless, we argue that the content (domain-invariant) features should be learned from images among all the domains. Consequently, each domain-specific content encoder of existing schemes fails to extract the domain-invariant features efficiently. To address this issue, we present a flexible and general SoloGAN model for efficient multimodal I2I translation among multiple domains with unpaired data. In contrast to existing methods, the SoloGAN algorithm uses a single projection discriminator with an additional auxiliary classifier, and shares the encoder and generator for all domains. As such, the SoloGAN model can be trained effectively with images from all domains such that the domain-invariant content representation can be efficiently extracted. Qualitative and quantitative results over a wide range of datasets against several counterparts and variants of the SoloGAN model demonstrate the merits of the method, especially for the challenging I2I translation tasks, i.e., tasks that involve extreme shape variations or need to keep the complex backgrounds unchanged after translations. Furthermore, we demonstrate the contribution of each component using ablation studies.

READ FULL TEXT

page 2

page 3

page 7

page 8

page 9

page 10

page 11

page 12

08/27/2020

Crossing-Domain Generative Adversarial Networks for Unsupervised Multi-Domain Image-to-Image Translation

State-of-the-art techniques in Generative Adversarial Networks (GANs) ha...
03/28/2018

Image Generation and Translation with Disentangled Representations

Generative models have made significant progress in the tasks of modelin...
11/24/2019

EDIT: Exemplar-Domain Aware Image-to-Image Translation

Image-to-image translation is to convert an image of the certain style t...
01/24/2019

Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation

Current approaches have made great progress on image-to-image translatio...
11/28/2021

Gated SwitchGAN for multi-domain facial image translation

Recent studies on multi-domain facial image translation have achieved im...
07/06/2021

Independent Encoder for Deep Hierarchical Unsupervised Image-to-Image Translation

The main challenges of image-to-image (I2I) translation are to make the ...
06/24/2020

Face-to-Music Translation Using a Distance-Preserving Generative Adversarial Network with an Auxiliary Discriminator

Learning a mapping between two unrelated domains-such as image and audio...