Revisiting Self-Supervised Visual Representation Learning

01/25/2019
by   Alexander Kolesnikov, et al.
Google
1

Unsupervised visual representation learning remains a largely unsolved problem in computer vision research. Among a big body of recently proposed approaches for unsupervised learning of visual representations, a class of self-supervised techniques achieves superior performance on many challenging benchmarks. A large number of the pretext tasks for self-supervised learning have been studied, but other important aspects, such as the choice of convolutional neural networks (CNN), has not received equal attention. Therefore, we revisit numerous previously proposed self-supervised models, conduct a thorough large scale study and, as a result, uncover multiple crucial insights. We challenge a number of common practices in selfsupervised visual representation learning and observe that standard recipes for CNN design do not always translate to self-supervised representation learning. As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin.

READ FULL TEXT
08/24/2020

Self-Supervised Learning for Large-Scale Unsupervised Image Clustering

Unsupervised learning has always been appealing to machine learning rese...
12/01/2020

Towards Good Practices in Self-supervised Representation Learning

Self-supervised representation learning has seen remarkable progress in ...
06/21/2021

Visual Probing: Cognitive Framework for Explaining Self-Supervised Image Representations

Recently introduced self-supervised methods for image representation lea...
06/03/2020

A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning

Probabilistic Latent Variable Models (LVMs) provide an alternative to se...
03/28/2021

Representation Learning by Ranking under multiple tasks

In recent years, representation learning has become the research focus o...
08/27/2019

Self-Supervised Representation Learning via Neighborhood-Relational Encoding

In this paper, we propose a novel self-supervised representation learnin...
08/28/2023

Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning

Ensembling a neural network is a widely recognized approach to enhance m...

Please sign up or login with your details

Forgot password? Click here to reset