MSR: Making Self-supervised learning Robust to Aggressive Augmentations

by   Yingbin Bai, et al.

Most recent self-supervised learning methods learn visual representation by contrasting different augmented views of images. Compared with supervised learning, more aggressive augmentations have been introduced to further improve the diversity of training pairs. However, aggressive augmentations may distort images' structures leading to a severe semantic shift problem that augmented views of the same image may not share the same semantics, thus degrading the transfer performance. To address this problem, we propose a new SSL paradigm, which counteracts the impact of semantic shift by balancing the role of weak and aggressively augmented pairs. Specifically, semantically inconsistent pairs are of minority and we treat them as noisy pairs. Note that deep neural networks (DNNs) have a crucial memorization effect that DNNs tend to first memorize clean (majority) examples before overfitting to noisy (minority) examples. Therefore, we set a relatively large weight for aggressively augmented data pairs at the early learning stage. With the training going on, the model begins to overfit noisy pairs. Accordingly, we gradually reduce the weights of aggressively augmented pairs. In doing so, our method can better embrace the aggressive augmentations and neutralize the semantic shift problem. Experiments show that our model achieves 73.1 with ResNet-50 for 200 epochs, which is a 2.5 experiments also demonstrate that the learned representations can transfer well for various downstream tasks.


GraVIS: Grouping Augmented Views from Independent Sources for Dermatology Analysis

Self-supervised representation learning has been extremely successful in...

Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning

In contrastive self-supervised learning, the common way to learn discrim...

Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods

Supervised training of deep neural networks on pairs of clean image and ...

Directional Self-supervised Learning for Risky Image Augmentations

Only a few cherry-picked robust augmentation policies are beneficial to ...

Additional Positive Enables Better Representation Learning for Medical Images

This paper presents a new way to identify additional positive pairs for ...

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-su...

Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning

Deep Neural Networks (DNNs) have achieved excellent performance in vario...

Please sign up or login with your details

Forgot password? Click here to reset