DeepAI AI Chat
Log In Sign Up

Momentum Contrast for Unsupervised Visual Representation Learning

11/13/2019
by   Kaiming He, et al.
25

We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/19/2021

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

Momentum Contrast (MoCo) achieves great success for unsupervised visual ...
07/13/2022

Unsupervised Visual Representation Learning by Synchronous Momentum Grouping

In this paper, we propose a genuine group-level contrastive visual repre...
03/09/2020

Improved Baselines with Momentum Contrastive Learning

Contrastive unsupervised learning has recently shown encouraging progres...
12/21/2020

Online Bag-of-Visual-Words Generation for Unsupervised Representation Learning

Learning image representations without human supervision is an important...
04/05/2023

Adaptive Data Augmentation for Contrastive Learning

In computer vision, contrastive learning is the most advanced unsupervis...
11/11/2022

Masked Contrastive Representation Learning

Masked image modelling (e.g., Masked AutoEncoder) and contrastive learni...
06/08/2021

Contrastive Representation Learning for Hand Shape Estimation

This work presents improvements in monocular hand shape estimation by bu...

Code Repositories

moco

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722


view repo

moco.tensorflow

A TensorFlow re-implementation of Momentum Contrast (MoCo): https://arxiv.org/abs/1911.05722


view repo

MoCo-TF

TF 2.x implementation of MoCo v1 (Momentum Contrast for Unsupervised Visual Representation Learning, CVPR 2020) and MoCo v2 (Improved Baselines with Momentum Contrastive Learning, 2020).


view repo