DeepAI AI Chat
Log In Sign Up

SpatialFlow: Bridging All Tasks for Panoptic Segmentation

by   Qiang Chen, et al.

The newly proposed panoptic segmentation task, which aims to encompass the tasks of instance segmentation (for things) and semantic segmentation (for stuff), is an essential step toward real-world vision systems and has attracted a lot of attention in the vision community. Recently, several works have been proposed for this task. Most of them focused on unifying two tasks by sharing the backbone but ignored to highlight the significance of fully interweaving features between tasks, such as providing the spatial context of objects to both semantic and instance segmentation. However, being well aware of locations of objects is fundamental to many vision tasks, e.g., object detection, instance segmentation, semantic segmentation. In this paper, we propose object spatial information flows, which can bridge all tasks together by delivering the spatial context from the box regression task to others. Based on these flows, we present a location-aware and unified framework for panoptic segmentation, SpatialFlow. The spatial information flows in SpatialFlow can provide clues for segmenting both things and stuff and help networks better understand the whole image. Moreover, instead of endowing Mask R-CNN with a stuff segmentation branch on a shared backbone, we design four parallel sub-networks for sub-tasks, which facilitate the feature integration among different tasks. We perform a detail ablation study on MS-COCO and Cityscapes panoptic benchmarks. Extensive experiments show that SpatialFlow achieves state-of-the-art results and can boost the performance of things and stuff segmentation at the same time.


page 6

page 10

page 11


Learning Panoptic Segmentation from Instance Contours

Panoptic Segmentation aims to provide an understanding of background (st...

Learning to Fuse Things and Stuff

We propose an end-to-end learning approach for panoptic segmentation, a ...

Single-shot Path Integrated Panoptic Segmentation

Panoptic segmentation, which is a novel task of unifying instance segmen...

CARAFE++: Unified Content-Aware ReAssembly of FEatures

Feature reassembly, i.e. feature downsampling and upsampling, is a key o...

Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton

In this paper, we solve three low-level pixel-wise vision problems, incl...

CARAFE: Content-Aware ReAssembly of FEatures

Feature upsampling is a key operation in a number of modern convolutiona...

Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey

From the autonomous car driving to medical diagnosis, the requirement of...

Code Repositories


SpatialFlow: Bridging all Task for Panoptic Segmentation. In TCSVT2020

view repo