Self-supervised Pretraining of Visual Features in the Wild

03/02/2021
by   Priya Goyal, et al.
0

Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and SwAV have reduced the gap with supervised methods. These results have been achieved in a control environment, that is the highly curated ImageNet dataset. However, the premise of self-supervised learning is that it can learn from any random image and from any unbounded dataset. In this work, we explore if self-supervision lives to its expectation by training large models on random, uncurated images with no supervision. Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2 1 Interestingly, we also observe that self-supervised models are good few-shot learners achieving 77.9 https://github.com/facebookresearch/vissl

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/28/2020

CompRess: Self-Supervised Learning by Compressing Representations

Self-supervised learning aims to learn good representations with unlabel...
12/04/2020

Super-Selfish: Self-Supervised Learning on Images with PyTorch

Super-Selfish is an easy to use PyTorch framework for image-based self-s...
08/01/2021

Self-supervised Learning with Local Attention-Aware Feature

In this work, we propose a novel methodology for self-supervised learnin...
08/20/2021

Self-supervised learning for joint SAR and multispectral land cover classification

Self-supervised learning techniques are gaining popularity due to their ...
11/23/2021

DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning

Self-supervised learning algorithms, including BERT and SimCLR, have ena...
11/30/2020

SIR: Self-supervised Image Rectification via Seeing the Same Scene from Multiple Different Lenses

Deep learning has demonstrated its power in image rectification by lever...
04/24/2020

Extending and Analyzing Self-Supervised Learning Across Domains

Self-supervised representation learning has achieved impressive results ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.