Unbalanced Feature Transport for Exemplar-based Image Translation

06/19/2021
by   Fangneng Zhan, et al.
0

Despite the great success of GANs in images translation with different conditioned inputs such as semantic segmentation and edge maps, generating high-fidelity realistic images with reference styles remains a grand challenge in conditional image-to-image translation. This paper presents a general image translation framework that incorporates optimal transport for feature alignment between conditional inputs and style exemplars in image translation. The introduction of optimal transport mitigates the constraint of many-to-one feature matching significantly while building up accurate semantic correspondences between conditional inputs and exemplars. We design a novel unbalanced optimal transport to address the transport between features with deviational distributions which exists widely between conditional inputs and exemplars. In addition, we design a semantic-activation normalization scheme that injects style features of exemplars into the image translation process successfully. Extensive experiments over multiple image translation tasks show that our method achieves superior image translation qualitatively and quantitatively as compared with the state-of-the-art.

READ FULL TEXT

page 3

page 4

page 5

page 7

page 8

07/07/2021

Bi-level Feature Alignment for Versatile Image Translation and Manipulation

Generative adversarial networks (GANs) have achieved great success in im...
04/12/2020

Cross-domain Correspondence Learning for Exemplar-based Image Translation

We present a general framework for exemplar-based image translation, whi...
04/01/2022

Marginal Contrastive Correspondence for Guided Image Generation

Exemplar-based image translation establishes dense correspondences betwe...
11/20/2021

Delving into Rectifiers in Style-Based Image Translation

While modern image translation techniques can create photorealistic synt...
05/27/2020

Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

Selecting input features of top relevance has become a popular method fo...
12/03/2021

Panoptic-based Object Style-Align for Image-to-Image Translation

Despite remarkable recent progress in image translation, the complex sce...
03/09/2021

Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images

Interactive simulation of ultrasound imaging greatly facilitates sonogra...