SpatialFlow: Bridging All Tasks for Panoptic Segmentation

10/19/2019
by   Qiang Chen, et al.
19

The newly proposed panoptic segmentation task, which aims to encompass the tasks of instance segmentation (for things) and semantic segmentation (for stuff), is an essential step toward real-world vision systems and has attracted a lot of attention in the vision community. Recently, several works have been proposed for this task. Most of them focused on unifying two tasks by sharing the backbone but ignored to highlight the significance of fully interweaving features between tasks, such as providing the spatial context of objects to both semantic and instance segmentation. However, being well aware of locations of objects is fundamental to many vision tasks, e.g., object detection, instance segmentation, semantic segmentation. In this paper, we propose object spatial information flows, which can bridge all tasks together by delivering the spatial context from the box regression task to others. Based on these flows, we present a location-aware and unified framework for panoptic segmentation, SpatialFlow. The spatial information flows in SpatialFlow can provide clues for segmenting both things and stuff and help networks better understand the whole image. Moreover, instead of endowing Mask R-CNN with a stuff segmentation branch on a shared backbone, we design four parallel sub-networks for sub-tasks, which facilitate the feature integration among different tasks. We perform a detail ablation study on MS-COCO and Cityscapes panoptic benchmarks. Extensive experiments show that SpatialFlow achieves state-of-the-art results and can boost the performance of things and stuff segmentation at the same time.

READ FULL TEXT

page 6

page 10

page 11

research
10/16/2020

Learning Panoptic Segmentation from Instance Contours

Panoptic Segmentation aims to provide an understanding of background (st...
research
12/04/2018

Learning to Fuse Things and Stuff

We propose an end-to-end learning approach for panoptic segmentation, a ...
research
12/03/2020

Single-shot Path Integrated Panoptic Segmentation

Panoptic segmentation, which is a novel task of unifying instance segmen...
research
04/06/2023

SegGPT: Segmenting Everything In Context

We present SegGPT, a generalist model for segmenting everything in conte...
research
01/13/2022

CFNet: Learning Correlation Functions for One-Stage Panoptic Segmentation

Recently, there is growing attention on one-stage panoptic segmentation ...
research
04/18/2020

Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton

In this paper, we solve three low-level pixel-wise vision problems, incl...
research
12/07/2020

CARAFE++: Unified Content-Aware ReAssembly of FEatures

Feature reassembly, i.e. feature downsampling and upsampling, is a key o...

Please sign up or login with your details

Forgot password? Click here to reset