Modulated Contrast for Versatile Image Synthesis

03/17/2022
by   Fangneng Zhan, et al.
0

Perceiving the similarity between images has been a long-standing and fundamental problem underlying various visual generation tasks. Predominant approaches measure the inter-image distance by computing pointwise absolute deviations, which tends to estimate the median of instance distributions and leads to blurs and artifacts in the generated images. This paper presents MoNCE, a versatile metric that introduces image contrast to learn a calibrated metric for the perception of multifaceted inter-image distances. Unlike vanilla contrast which indiscriminately pushes negative samples from the anchor regardless of their similarity, we propose to re-weight the pushing force of negative samples adaptively according to their similarity to the anchor, which facilitates the contrastive learning from informative negative samples. Since multiple patch-level contrastive objectives are involved in image distance measurement, we introduce optimal transport in MoNCE to modulate the pushing force of negative samples collaboratively across multiple contrastive objectives. Extensive experiments over multiple image translation tasks show that the proposed MoNCE outperforms various prevailing metrics substantially. The code is available at https://github.com/fnzhan/MoNCE.

READ FULL TEXT

page 1

page 6

page 7

research
04/06/2023

Synthetic Hard Negative Samples for Contrastive Learning

Contrastive learning has emerged as an essential approach for self-super...
research
06/21/2023

Inter-Instance Similarity Modeling for Contrastive Learning

The existing contrastive learning methods widely adopt one-hot instance ...
research
02/26/2022

Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding

Contrastive learning is emerging as a powerful technique for extracting ...
research
11/26/2021

Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning

The goal of contrastive learning based pre-training is to leverage large...
research
10/27/2021

Robust Contrastive Learning Using Negative Samples with Diminished Semantics

Unsupervised learning has recently made exceptional progress because of ...
research
03/28/2022

Attributable Visual Similarity Learning

This paper proposes an attributable visual similarity learning (AVSL) fr...
research
07/06/2023

Contrast Is All You Need

In this study, we analyze data-scarce classification scenarios, where av...

Please sign up or login with your details

Forgot password? Click here to reset