GOSS: Towards Generalized Open-set Semantic Segmentation

03/23/2022
by   Jie Hong, et al.
0

In this paper, we present and study a new image segmentation task, called Generalized Open-set Semantic Segmentation (GOSS). Previously, with the well-known open-set semantic segmentation (OSS), the intelligent agent only detects the unknown regions without further processing, limiting their perception of the environment. It stands to reason that a further analysis of the detected unknown pixels would be beneficial. Therefore, we propose GOSS, which unifies the abilities of two well-defined segmentation tasks, OSS and generic segmentation (GS), in a holistic way. Specifically, GOSS classifies pixels as belonging to known classes, and clusters (or groups) of pixels of unknown class are labelled as such. To evaluate this new expanded task, we further propose a metric which balances the pixel classification and clustering aspects. Moreover, we build benchmark tests on top of existing datasets and propose a simple neural architecture as a baseline, which jointly predicts pixel classification and clustering under open-set settings. Our experiments on multiple benchmarks demonstrate the effectiveness of our baseline. We believe our new GOSS task can produce an expressive image understanding for future research. Code will be made available.

READ FULL TEXT

page 1

page 5

page 9

research
06/11/2021

KRADA: Known-region-aware Domain Alignment for Open World Semantic Segmentation

In semantic segmentation, we aim to train a pixel-level classifier to as...
research
05/18/2021

Exemplar-Based Open-Set Panoptic Segmentation Network

We extend panoptic segmentation to the open-world and introduce an open-...
research
06/25/2020

Fully Convolutional Open Set Segmentation

In semantic segmentation knowing about all existing classes is essential...
research
11/24/2016

Recalling Holistic Information for Semantic Segmentation

Semantic segmentation requires a detailed labeling of image pixels by ob...
research
03/21/2023

Influencer Backdoor Attack on Semantic Segmentation

When a small number of poisoned samples are injected into the training d...
research
11/25/2022

Pixels Together Strong: Segmenting Unknown Regions Rejected by All

Semantic segmentation methods typically perform per-pixel classification...
research
09/12/2022

Holistic Segmentation

As panoptic segmentation provides a prediction for every pixel in input,...

Please sign up or login with your details

Forgot password? Click here to reset