DeepAI AI Chat
Log In Sign Up

Self-Supervised Representation Learning: Introduction, Advances and Challenges

by   Linus Ericsson, et al.

Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today. These methods have advanced rapidly in recent years, with their efficacy approaching and sometimes surpassing fully supervised pre-training alternatives across a variety of data modalities including image, video, sound, text and graphs. This article introduces this vibrant area including key concepts, the four main families of approach and associated state of the art, and how self-supervised methods are applied to diverse modalities of data. We further discuss practical considerations including workflows, representation transferability, and compute cost. Finally, we survey the major open challenges in the field that provide fertile ground for future work.


Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data

Recently, Self-Supervised Representation Learning (SSRL) has attracted m...

Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works

Deep learning has achieved great success in learning features from massi...

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting

Self-supervised learning has gained prominence due to its efficacy at le...

Self-Supervised Representation Learning on Document Images

This work analyses the impact of self-supervised pre-training on documen...

How Well Do Self-Supervised Methods Perform in Cross-Domain Few-Shot Learning?

Cross-domain few-shot learning (CDFSL) remains a largely unsolved proble...

Survey on Self-supervised Representation Learning Using Image Transformations

Deep neural networks need huge amount of training data, while in real wo...

A Survey of Traversability Estimation for Mobile Robots

Traversability illustrates the difficulty of driving through a specific ...