Log In Sign Up

QueryInst: Parallelly Supervised Mask Query for Instance Segmentation

by   Yuxin Fang, et al.

Recently, query based object detection frameworks achieve comparable performance with previous state-of-the-art object detectors. However, how to fully leverage such frameworks to perform instance segmentation remains an open problem. In this paper, we present QueryInst, a query based instance segmentation method driven by parallel supervision on dynamic mask heads. The key insight of QueryInst is to leverage the intrinsic one-to-one correspondence in object queries across different stages, as well as one-to-one correspondence between mask RoI features and object queries in the same stage. This approach eliminates the explicit multi-stage mask head connection and the proposal distribution inconsistency issues inherent in non-query based multi-stage instance segmentation methods. We conduct extensive experiments on three challenging benchmarks, i.e., COCO, CityScapes, and YouTube-VIS to evaluate the effectiveness of QueryInst in instance segmentation and video instance segmentation (VIS) task. Specifically, using ResNet-101-FPN backbone, QueryInst obtains 48.1 box AP and 42.8 mask AP on COCO test-dev, which is 2 points higher than HTC in terms of both box AP and mask AP, while runs 2.4 times faster. For video instance segmentation, QueryInst achieves the best performance among all online VIS approaches and strikes a decent speed-accuracy trade-off. Code is available at <>.


page 8

page 10

page 11

page 12


Mask Encoding for Single Shot Instance Segmentation

To date, instance segmentation is dominated by twostage methods, as pion...

ISTR: End-to-End Instance Segmentation with Transformers

End-to-end paradigms significantly improve the accuracy of various deep-...

SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation

Single-stage instance segmentation approaches have recently gained popul...

SCNet: Training Inference Sample Consistency for Instance Segmentation

Cascaded architectures have brought significant performance improvement ...

Tracking Instances as Queries

Recently, query based deep networks catch lots of attention owing to the...

YolactEdge: Real-time Instance Segmentation on the Edge (Jetson AGX Xavier: 30 FPS, RTX 2080 Ti: 170 FPS)

We propose YolactEdge, the first competitive instance segmentation appro...

Temporally Efficient Vision Transformer for Video Instance Segmentation

Recently vision transformer has achieved tremendous success on image-lev...

Code Repositories


Instances as Queries (ICCV 2021)

view repo