Toward Multimodal Image-to-Image Translation

11/30/2017
by   Jun-Yan Zhu, et al.
0

Many image-to-image translation problems are ambiguous, as a single input image may correspond to multiple possible outputs. In this work, we aim to model a distribution of possible outputs in a conditional generative modeling setting. The ambiguity of the mapping is distilled in a low-dimensional latent vector, which can be randomly sampled at test time. A generator learns to map the given input, combined with this latent code, to the output. We explicitly encourage the connection between output and the latent code to be invertible. This helps prevent a many-to-one mapping from the latent code to the output during training, also known as the problem of mode collapse, and produces more diverse results. We explore several variants of this approach by employing different training objectives, network architectures, and methods of injecting the latent code. Our proposed method encourages bijective consistency between the latent encoding and output modes. We present a systematic comparison of our method and other variants on both perceptual realism and diversity.

READ FULL TEXT

page 2

page 3

page 7

page 9

research
10/11/2018

SingleGAN: Image-to-Image Translation by a Single-Generator Network using Multiple Generative Adversarial Learning

Image translation is a burgeoning field in computer vision where the goa...
research
08/19/2019

SDIT: Scalable and Diverse Cross-domain Image Translation

Recently, image-to-image translation research has witnessed remarkable p...
research
06/26/2018

Multi-Mapping Image-to-Image Translation with Central Biasing Normalization

Recent image-to-image translation tasks attempt to extend the model from...
research
09/28/2019

Semantic Example Guided Image-to-Image Translation

Many image-to-image (I2I) translation problems are in nature of high div...
research
06/29/2020

Simplifying Models with Unlabeled Output Data

We focus on prediction problems with high-dimensional outputs that are s...
research
03/09/2021

Generative Transition Mechanism to Image-to-Image Translation via Encoded Transformation

In this paper, we revisit the Image-to-Image (I2I) translation problem w...
research
08/02/2018

Diverse Image-to-Image Translation via Disentangled Representations

Image-to-image translation aims to learn the mapping between two visual ...

Please sign up or login with your details

Forgot password? Click here to reset