Learning Universal Shape Dictionary for Realtime Instance Segmentation

by   Tutian Tang, et al.

We present a novel explicit shape representation for instance segmentation. Based on how to model the object shape, current instance segmentation systems can be divided into two categories, implicit and explicit models. The implicit methods, which represent the object mask/contour by intractable network parameters, and produce it through pixel-wise classification, are predominant. However, the explicit methods, which parameterize the shape with simple and explainable models, are less explored. Since the operations to generate the final shape are light-weighted, the explicit methods have a clear speed advantage over implicit methods, which is crucial for real-world applications. The proposed USD-Seg adopts a linear model, sparse coding with dictionary, for object shapes. First, it learns a dictionary from a large collection of shape datasets, making any shape being able to be decomposed into a linear combination through the dictionary. Hence the name "Universal Shape Dictionary". Then it adds a simple shape vector regression head to ordinary object detector, giving the detector segmentation ability with minimal overhead. For quantitative evaluation, we use both average precision (AP) and the proposed Efficiency of AP (AP_E) metric, which intends to also measure the computational consumption of the framework to cater to the requirements of real-world applications. We report experimental results on the challenging COCO dataset, in which our single model on a single Titan Xp GPU achieves 35.8 AP and 27.8 AP_E at 65 fps with YOLOv4 as base detector, 34.1 AP and 28.6 AP_E at 12 fps with FCOS as base detector.


page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8


CenterMask: single shot instance segmentation with point representation

In this paper, we propose a single-shot instance segmentation method, wh...

Explicit Shape Encoding for Real-Time Instance Segmentation

In this paper, we propose a novel top-down instance segmentation framewo...

Single Pixel Reconstruction for One-stage Instance Segmentation

Object instance segmentation is one of the most fundamental but challeng...

ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

Instance segmentation aims to detect and segment individual objects in a...

Mask-aware IoU for Anchor Assignment in Real-time Instance Segmentation

This paper presents Mask-aware Intersection-over-Union (maIoU) for assig...

Iterative Instance Segmentation

Existing methods for pixel-wise labelling tasks generally disregard the ...

In Defense of Online Models for Video Instance Segmentation

In recent years, video instance segmentation (VIS) has been largely adva...