DeepAI AI Chat
Log In Sign Up

MixCo: Mix-up Contrastive Learning for Visual Representation

10/13/2020
by   Sungnyun Kim, et al.
1

Contrastive learning has shown remarkable results in recent self-supervised approaches for visual representation. By learning to contrast positive pairs' representation from the corresponding negatives pairs, one can train good visual representations without human annotations. This paper proposes Mix-up Contrast (MixCo), which extends the contrastive learning concept to semi-positives encoded from the mix-up of positive and negative images. MixCo aims to learn the relative similarity of representations, reflecting how much the mixed images have the original positives. We validate the efficacy of MixCo when applied to the recent self-supervised learning algorithms under the standard linear evaluation protocol on TinyImageNet, CIFAR10, and CIFAR100. In the experiments, MixCo consistently improves test accuracy. Remarkably, the improvement is more significant when the learning capacity (e.g., model size) is limited, suggesting that MixCo might be more useful in real-world scenarios.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/28/2022

Non-contrastive approaches to similarity learning: positive examples are all you need

The similarity learning problem in the oil & gas industry aims to constr...
11/23/2020

Boosting Contrastive Self-Supervised Learning with False Negative Cancellation

Self-supervised representation learning has witnessed significant leaps ...
06/28/2023

Semantic Positive Pairs for Enhancing Contrastive Instance Discrimination

Self-supervised learning algorithms based on instance discrimination eff...
08/06/2021

Improving Contrastive Learning by Visualizing Feature Transformation

Contrastive learning, which aims at minimizing the distance between posi...
06/20/2023

Understanding Contrastive Learning Through the Lens of Margins

Self-supervised learning, or SSL, holds the key to expanding the usage o...
02/14/2022

Learn by Challenging Yourself: Contrastive Visual Representation Learning with Hard Sample Generation

Contrastive learning (CL), a self-supervised learning approach, can effe...
01/05/2023

Learning by Sorting: Self-supervised Learning with Group Ordering Constraints

Contrastive learning has become a prominent ingredient in learning repre...