BoxInst: High-Performance Instance Segmentation with Box Annotations

by   Zhi Tian, et al.

We present a high-performance method that can achieve mask-level instance segmentation with only bounding-box annotations for training. While this setting has been studied in the literature, here we show significantly stronger performance with a simple design (e.g., dramatically improving previous best reported mask AP of 21.1 Our core idea is to redesign the loss of learning masks in instance segmentation, with no modification to the segmentation network itself. The new loss functions can supervise the mask training without relying on mask annotations. This is made possible with two loss terms, namely, 1) a surrogate term that minimizes the discrepancy between the projections of the ground-truth box and the predicted mask; 2) a pairwise loss that can exploit the prior that proximal pixels with similar colors are very likely to have the same category label. Experiments demonstrate that the redesigned mask loss can yield surprisingly high-quality instance masks with only box annotations. For example, without using any mask annotations, with a ResNet-101 backbone and 3x training schedule, we achieve 33.2 of the fully supervised counterpart). Our excellent experiment results on COCO and Pascal VOC indicate that our method dramatically narrows the performance gap between weakly and fully supervised instance segmentation. Code is available at:


page 1

page 2

page 8


Mask Encoding for Single Shot Instance Segmentation

To date, instance segmentation is dominated by twostage methods, as pion...

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

Binary grid mask representation is broadly used in instance segmentation...

GAN Mask R-CNN:Instance semantic segmentation benefits from generativeadversarial networks

In designing instance segmentation ConvNets that reconstruct masks, segm...

Box-supervised Instance Segmentation with Level Set Evolution

In contrast to the fully supervised methods using pixel-wise mask labels...

The surprising impact of mask-head architecture on novel class segmentation

Instance segmentation models today are very accurate when trained on lar...

Learning Segmentation Masks with the Independence Prior

An instance with a bad mask might make a composite image that uses it lo...

Utilizing Mask R-CNN for Waterline Detection in Canoe Sprint Video Analysis

Determining a waterline in images recorded in canoe sprint training is a...