Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation

04/23/2022
by   Yupei Lin, et al.
0

Unpaired image-to-image translation aims to find a mapping between the source domain and the target domain. To alleviate the problem of the lack of supervised labels for the source images, cycle-consistency based methods have been proposed for image structure preservation by assuming a reversible relationship between unpaired images. However, this assumption only uses limited correspondence between image pairs. Recently, contrastive learning (CL) has been used to further investigate the image correspondence in unpaired image translation by using patch-based positive/negative learning. Patch-based contrastive routines obtain the positives by self-similarity computation and recognize the rest patches as negatives. This flexible learning paradigm obtains auxiliary contextualized information at a low cost. As the negatives own an impressive sample number, with curiosity, we make an investigation based on a question: are all negatives necessary for feature contrastive learning? Unlike previous CL approaches that use negatives as much as possible, in this paper, we study the negatives from an information-theoretic perspective and introduce a new negative Pruning technology for Unpaired image-to-image Translation (PUT) by sparsifying and ranking the patches. The proposed algorithm is efficient, flexible and enables the model to learn essential information between corresponding patches stably. By putting quality over quantity, only a few negative patches are required to achieve better results. Lastly, we validate the superiority, stability, and versatility of our model through comparative experiments.

READ FULL TEXT

page 1

page 3

page 4

page 6

page 8

page 9

research
07/30/2020

Contrastive Learning for Unpaired Image-to-Image Translation

In image-to-image translation, each patch in the output should reflect t...
research
03/10/2023

Adaptive Supervised PatchNCE Loss for Learning H E-to-IHC Stain Translation with Inconsistent Groundtruth Image Pairs

Immunohistochemical (IHC) staining highlights the molecular information ...
research
04/15/2021

Dual Contrastive Learning for Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation tasks aim to find a mapping betw...
research
09/25/2021

Contrastive Unpaired Translation using Focal Loss for Patch Classification

Image-to-image translation models transfer images from input domain to o...
research
08/10/2021

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Contrastive learning shows great potential in unpaired image-to-image tr...
research
04/22/2023

Spectral normalized dual contrastive regularization for image-to-image translation

Existing image-to-image(I2I) translation methods achieve state-of-the-ar...
research
09/04/2020

SketchPatch: Sketch Stylization via Seamless Patch-level Synthesis

The paradigm of image-to-image translation is leveraged for the benefit ...

Please sign up or login with your details

Forgot password? Click here to reset