Multimodal Unsupervised Image-to-Image Translation

04/12/2018
by   Xun Huang, et al.
0

Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any pairs of corresponding images. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to the state-of-the-art approaches further demonstrates the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at https://github.com/nvlabs/MUNIT.

READ FULL TEXT

page 11

page 12

page 14

page 15

research
11/29/2018

Unsupervised Image-to-Image Translation Using Domain-Specific Variational Information Bound

Unsupervised image-to-image translation is a class of computer vision pr...
research
10/27/2021

Separating Content and Style for Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation aims to learn the mapping betwee...
research
05/28/2018

Exemplar Guided Unsupervised Image-to-Image Translation

Image-to-image translation task has become a popular topic recently. Mos...
research
11/19/2021

Global and Local Alignment Networks for Unpaired Image-to-Image Translation

The goal of unpaired image-to-image translation is to produce an output ...
research
06/22/2021

Fine-Tuning StyleGAN2 For Cartoon Face Generation

Recent studies have shown remarkable success in the unsupervised image t...
research
11/26/2021

ManiFest: Manifold Deformation for Few-shot Image Translation

Most image-to-image translation methods require a large number of traini...
research
06/26/2023

Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translation

This paper studies a novel energy-based cooperative learning framework f...

Please sign up or login with your details

Forgot password? Click here to reset