A Survey on Contrastive Self-supervised Learning

10/31/2020
by   Ashish Jaiswal, et al.
78

Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. It is capable of adopting self-defined pseudo labels as supervision and use the learned representations for several downstream tasks. Specifically, contrastive learning has recently become a dominant component in self-supervised learning methods for computer vision, natural language processing (NLP), and other domains. It aims at embedding augmented versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks in a contrastive learning setup, followed by different architectures that have been proposed so far. Next, we have a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make substantial progress.

READ FULL TEXT

page 2

page 4

page 5

page 7

page 9

page 13

research
06/15/2020

Self-supervised Learning: Generative or Contrastive

Deep supervised learning has achieved great success in the last decade. ...
research
02/16/2019

Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey

Large-scale labeled data are generally required to train deep neural net...
research
09/15/2021

SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations

While contrastive learning is proven to be an effective training strateg...
research
07/31/2022

COCOA: Cross Modality Contrastive Learning for Sensor Data

Self-Supervised Learning (SSL) is a new paradigm for learning discrimina...
research
05/23/2023

Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training

Although supervised learning has been highly successful in improving the...
research
09/14/2023

Hodge-Aware Contrastive Learning

Simplicial complexes prove effective in modeling data with multiway depe...
research
07/30/2023

Self-Supervised Learning of Gait-Based Biomarkers

Markerless motion capture (MMC) is revolutionizing gait analysis in clin...

Please sign up or login with your details

Forgot password? Click here to reset