Mix and match networks: encoder-decoder alignment for zero-pair image translation

04/06/2018
by   Yaxing Wang, et al.
0

We address the problem of image translation between domains or modalities for which no direct paired data is available (i.e. zero-pair translation). We propose mix and match networks, based on multiple encoders and decoders aligned in such a way that other encoder-decoder pairs can be composed at test time to perform unseen image translation tasks between domains or modalities for which explicit paired samples were not seen during training. We study the impact of autoencoders, side information and losses in improving the alignment and transferability of trained pairwise translation models to unseen translations. We show our approach is scalable and can perform colorization and style transfer between unseen combinations of domains. We evaluate our system in a challenging cross-modal setting where semantic segmentation is estimated from depth images, without explicit access to any depth-semantic segmentation training pairs. Our model outperforms baselines based on pix2pix and CycleGAN models.

READ FULL TEXT

page 5

page 7

page 8

research
03/08/2019

Mix and match networks: multi-domain alignment for unpaired image-to-image translation

This paper addresses the problem of inferring unseen cross-domain and cr...
research
11/11/2020

Zero-Pair Image to Image Translation using Domain Conditional Normalization

In this paper, we propose an approach based on domain conditional normal...
research
09/23/2022

Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs

A self-driving car must be able to reliably handle adverse weather condi...
research
11/01/2018

Examining Performance of Sketch-to-Image Translation Models with Multiclass Automatically Generated Paired Training Data

Image translation is a computer vision task that involves translating on...
research
11/13/2017

Zero-Shot Style Transfer in Text Using Recurrent Neural Networks

Zero-shot translation is the task of translating between a language pair...
research
08/31/2020

Langevin Cooling for Domain Translation

Domain translation is the task of finding correspondence between two dom...
research
07/26/2021

A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translation

Cross-contrast image translation is an important task for completing mis...

Please sign up or login with your details

Forgot password? Click here to reset