DeepAI AI Chat
Log In Sign Up

SpatialFlow: Bridging All Tasks for Panoptic Segmentation

10/19/2019
by   Qiang Chen, et al.
19

The newly proposed panoptic segmentation task, which aims to encompass the tasks of instance segmentation (for things) and semantic segmentation (for stuff), is an essential step toward real-world vision systems and has attracted a lot of attention in the vision community. Recently, several works have been proposed for this task. Most of them focused on unifying two tasks by sharing the backbone but ignored to highlight the significance of fully interweaving features between tasks, such as providing the spatial context of objects to both semantic and instance segmentation. However, being well aware of locations of objects is fundamental to many vision tasks, e.g., object detection, instance segmentation, semantic segmentation. In this paper, we propose object spatial information flows, which can bridge all tasks together by delivering the spatial context from the box regression task to others. Based on these flows, we present a location-aware and unified framework for panoptic segmentation, SpatialFlow. The spatial information flows in SpatialFlow can provide clues for segmenting both things and stuff and help networks better understand the whole image. Moreover, instead of endowing Mask R-CNN with a stuff segmentation branch on a shared backbone, we design four parallel sub-networks for sub-tasks, which facilitate the feature integration among different tasks. We perform a detail ablation study on MS-COCO and Cityscapes panoptic benchmarks. Extensive experiments show that SpatialFlow achieves state-of-the-art results and can boost the performance of things and stuff segmentation at the same time.

READ FULL TEXT

page 6

page 10

page 11

10/16/2020

Learning Panoptic Segmentation from Instance Contours

Panoptic Segmentation aims to provide an understanding of background (st...
12/04/2018

Learning to Fuse Things and Stuff

We propose an end-to-end learning approach for panoptic segmentation, a ...
12/03/2020

Single-shot Path Integrated Panoptic Segmentation

Panoptic segmentation, which is a novel task of unifying instance segmen...
12/07/2020

CARAFE++: Unified Content-Aware ReAssembly of FEatures

Feature reassembly, i.e. feature downsampling and upsampling, is a key o...
04/18/2020

Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton

In this paper, we solve three low-level pixel-wise vision problems, incl...
05/06/2019

CARAFE: Content-Aware ReAssembly of FEatures

Feature upsampling is a key operation in a number of modern convolutiona...
01/13/2020

Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey

From the autonomous car driving to medical diagnosis, the requirement of...

Code Repositories

SpatialFlow

SpatialFlow: Bridging all Task for Panoptic Segmentation. In TCSVT2020


view repo