Dense Semantic Contrast for Self-Supervised Visual Representation Learning

09/16/2021
by   Xiaoni Li, et al.
0

Self-supervised representation learning for visual pre-training has achieved remarkable success with sample (instance or pixel) discrimination and semantics discovery of instance, whereas there still exists a non-negligible gap between pre-trained model and downstream dense prediction tasks. Concretely, these downstream tasks require more accurate representation, in other words, the pixels from the same object must belong to a shared semantic category, which is lacking in the previous methods. In this work, we present Dense Semantic Contrast (DSC) for modeling semantic category decision boundaries at a dense level to meet the requirement of these tasks. Furthermore, we propose a dense cross-image semantic contrastive learning framework for multi-granularity representation learning. Specially, we explicitly explore the semantic structure of the dataset by mining relations among pixels from different perspectives. For intra-image relation modeling, we discover pixel neighbors from multiple views. And for inter-image relations, we enforce pixel representation from the same semantic class to be more similar than the representation from different classes in one mini-batch. Experimental results show that our DSC model outperforms state-of-the-art methods when transferring to downstream dense prediction tasks, including object detection, semantic segmentation, and instance segmentation. Code will be made available.

READ FULL TEXT

page 1

page 3

page 8

research
11/18/2020

Dense Contrastive Learning for Self-Supervised Visual Pre-Training

To date, most existing self-supervised learning methods are designed and...
research
07/19/2021

Exploring Set Similarity for Dense Self-supervised Representation Learning

By considering the spatial correspondence, dense self-supervised represe...
research
11/15/2021

iBOT: Image BERT Pre-Training with Online Tokenizer

The success of language Transformers is primarily attributed to the pret...
research
05/30/2022

Self-Supervised Visual Representation Learning with Semantic Grouping

In this paper, we tackle the problem of learning visual representations ...
research
08/14/2023

Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning

Due to the flexible representation of arbitrary-shaped scene text and si...
research
03/29/2022

Self-Supervised Image Representation Learning with Geometric Set Consistency

We propose a method for self-supervised image representation learning un...
research
03/03/2021

Deep Clustering by Semantic Contrastive Learning

Whilst contrastive learning has achieved remarkable success in self-supe...

Please sign up or login with your details

Forgot password? Click here to reset