Synthetic Hard Negative Samples for Contrastive Learning

04/06/2023
by   Hengkui Dong, et al.
0

Contrastive learning has emerged as an essential approach for self-supervised learning in computer vision. The central objective of contrastive learning is to maximize the similarities between two augmented versions of the same image (positive pairs), while minimizing the similarities between different images (negative pairs). Recent studies have demonstrated that harder negative samples, i.e., those that are difficult to distinguish from anchor sample, play a more critical role in contrastive learning. In this paper, we propose a novel featurelevel method, namely sampling synthetic hard negative samples for contrastive learning (SSCL), to exploit harder negative samples more effectively. Specifically, 1) we generate more and harder negative samples by mixing negative samples, and then sample them by controlling the contrast of anchor sample with the other negative samples. 2) Considering that the negative samples obtained by sampling may have the problem of false negative samples, we further debias the negative samples. Our proposed method improves the classification performance on different image datasets and can be readily applied to existing methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/01/2022

Generating Counterfactual Hard Negative Samples for Graph Contrastive Learning

Graph contrastive learning has emerged as a powerful tool for unsupervis...
research
08/31/2022

Supervised Contrastive Learning with Hard Negative Samples

Unsupervised contrastive learning (UCL) is a self-supervised learning te...
research
01/27/2023

Bayesian Self-Supervised Contrastive Learning

Recent years have witnessed many successful applications of contrastive ...
research
08/13/2022

Enhancing Graph Contrastive Learning with Node Similarity

Graph Neural Networks (GNNs) have achieved great success in learning gra...
research
03/17/2022

Modulated Contrast for Versatile Image Synthesis

Perceiving the similarity between images has been a long-standing and fu...
research
10/02/2020

Hard Negative Mixing for Contrastive Learning

Contrastive learning has become a key component of self-supervised learn...
research
07/16/2022

Model-Aware Contrastive Learning: Towards Escaping Uniformity-Tolerance Dilemma in Training

Instance discrimination contrastive learning (CL) has achieved significa...

Please sign up or login with your details

Forgot password? Click here to reset