Novel Class Discovery in Semantic Segmentation

12/03/2021
by   Yuyang Zhao, et al.
0

We introduce a new setting of Novel Class Discovery in Semantic Segmentation (NCDSS), which aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In contrast to existing approaches that look at novel class discovery in image classification, we focus on the more challenging semantic segmentation. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image, which increases the difficulty in using the unlabeled data. To tackle this new setting, we leverage the labeled base data and a saliency model to coarsely cluster novel classes for model training in our basic framework. Additionally, we propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels, further improving the model performance on the novel classes. Our EUMS utilizes an entropy ranking technique and a dynamic reassignment to distill clean labels, thereby making full use of the noisy data via self-supervised learning. We build the NCDSS benchmark on the PASCAL-5^i dataset. Extensive experiments demonstrate the feasibility of the basic framework (achieving an average mIoU of 49.81 framework by 9.28

READ FULL TEXT

page 1

page 3

page 8

page 9

research
07/11/2022

Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations

In this paper, we show that recent advances in self-supervised feature l...
research
12/06/2020

Self-Training for Class-Incremental Semantic Segmentation

We study incremental learning for semantic segmentation where when learn...
research
09/19/2022

A Closer Look at Novel Class Discovery from the Labeled Set

Novel class discovery (NCD) aims to infer novel categories in an unlabel...
research
06/06/2023

Supervised Knowledge May Hurt Novel Class Discovery Performance

Novel class discovery (NCD) aims to infer novel categories in an unlabel...
research
12/06/2016

Diverse Sampling for Self-Supervised Learning of Semantic Segmentation

We propose an approach for learning category-level semantic segmentation...
research
07/16/2020

Efficient Full Image Interactive Segmentation by Leveraging Within-image Appearance Similarity

We propose a new approach to interactive full-image semantic segmentatio...
research
06/29/2021

Open-Set Representation Learning through Combinatorial Embedding

Visual recognition tasks are often limited to dealing with a small subse...

Please sign up or login with your details

Forgot password? Click here to reset