DeepAI AI Chat
Log In Sign Up

Constraining Multi-scale Pairwise Features between Encoder and Decoder Using Contrastive Learning for Unpaired Image-to-Image Translation

by   Xiuding Cai, et al.
University of the Chinese Academy of Sciences

Contrastive learning (CL) has shown great potential in image-to-image translation (I2I). Current CL-based I2I methods usually re-exploit the encoder of the generator to maximize the mutual information between the input and generated images, which does not exert an active effect on the decoder part. In addition, though negative samples play a crucial role in CL, most existing methods adopt a random sampling strategy, which may be less effective. In this paper, we rethink the CL paradigm in the unpaired I2I tasks from two perspectives and propose a new one-sided image translation framework called EnCo. First, we present an explicit constraint on the multi-scale pairwise features between the encoder and decoder of the generator to guarantee the semantic consistency of the input and generated images. Second, we propose a discriminative attention-guided negative sampling strategy to replace the random negative sampling, which significantly improves the performance of the generative model with an almost negligible computational overhead. Compared with existing methods, EnCo acts more effective and efficient. Extensive experiments on several popular I2I datasets demonstrate the effectiveness and advantages of our proposed approach, and we achieve several state-of-the-art compared to previous methods.


page 1

page 2

page 6

page 10

page 13

page 14

page 15

page 16


Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Contrastive learning shows great potential in unpaired image-to-image tr...

Dual Contrastive Learning for Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation tasks aim to find a mapping betw...

Less is More: Unified Model for Unsupervised Multi-Domain Image-to-Image Translation

In this paper, we aim at solving the multi-domain image-to-image transla...

Contrastive Learning for Unpaired Image-to-Image Translation

In image-to-image translation, each patch in the output should reflect t...

The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation is an inherently ill-posed probl...

Multi-scale Self-calibrated Network for Image Light Source Transfer

Image light source transfer (LLST), as the most challenging task in the ...