YOLACT++: Better Real-time Instance Segmentation

12/03/2019
by   Daniel Bolya, et al.
0

We present a simple, fully-convolutional model for real-time (>30 fps) instance segmentation that achieves competitive results on MS COCO evaluated on a single Titan Xp, which is significantly faster than any previous state-of-the-art approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. We also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty. Finally, by incorporating deformable convolutions into the backbone network, optimizing the prediction head with better anchor scales and aspect ratios, and adding a novel fast mask re-scoring branch, our YOLACT++ model can achieve 34.1 mAP on MS COCO at 33.5 fps, which is fairly close to the state-of-the-art approaches while still running at real-time.

READ FULL TEXT
research
04/04/2019

YOLACT: Real-time Instance Segmentation

We present a simple, fully-convolutional model for real-time instance se...
research
12/03/2019

SAIS: Single-stage Anchor-free Instance Segmentation

In this paper, we propose a simple yet efficientinstance segmentation ap...
research
03/23/2020

EPSNet: Efficient Panoptic Segmentation Network with Cross-layer Attention Fusion

Panoptic segmentation is a scene parsing task which unifies semantic seg...
research
09/19/2018

Faster Training of Mask R-CNN by Focusing on Instance Boundaries

We present an auxiliary task to Mask R-CNN, an instance segmentation net...
research
02/07/2020

FourierNet: Compact mask representation for instance segmentation using differentiable shape decoders

We present FourierNet a single shot, anchor-free, fully convolutional in...
research
06/23/2021

Real-time Instance Segmentation with Discriminative Orientation Maps

Although instance segmentation has made considerable advancement over re...
research
02/06/2023

PatchDCT: Patch Refinement for High Quality Instance Segmentation

High-quality instance segmentation has shown emerging importance in comp...

Please sign up or login with your details

Forgot password? Click here to reset