The top performing object detectors and segmenters often follow a two-stage paradigm. They consist of a fully convolutional network, region proposal network (RPN), to perform dense prediction of the most likely regions of interest (RoIs). A set of light-weight networks, a.k.a. heads, are applied to re-align the features of RoIs and generate predictions . The quality and speed for mask generation is strongly tied to the structure of the mask heads. In addition, it is difficult for independent heads to share features with related tasks such as semantic segmentation which causes trouble for network architecture optimization.
Recent advances in one-stage object detection prove that one-stage methods such as FCOS can outperform their two-stage counterparts in accuracy . Enabling such one-stage detection frameworks to perform dense instance segmentation is highly desirable as 1) models consisting of only conventional operations are simpler and easier for cross-platform deployment; 2) a unified framework provides convenience and flexibility for multi-task network architecture optimization.
Dense instance segmenters can date back to DeepMask
, a top-down approach which generates dense instance masks with a sliding window. The representation of mask is encoded into a one-dimensional vector at each spatial location. Albeit being simple in structure, it has several obstacles in training that prevent it from achieving superior performance: 1) local-coherence between features and masks is lost; 2) the feature representation is redundant because a mask is repeatedly encoded at each foreground feature; 3) position information is degraded after downsampling with strided convolutions.
The first issue was studied by Dai et al. , who attempt to retain local-coherence by keeping multiple position-sensitive maps. This idea has been explored to its limits by Chen et al. , who proposes a dense aligned representation for each location of the target instance mask. However, this approach trades representation efficiency for alignment, making the second issue difficult to resolve. The third issue prevents heavily downsampled features to provide detailed instance information.
Recognizing these difficulties, a line of research takes a bottom-up strategy [1, 19, 20]. These methods generate dense per-pixel embedding features and use some techniques to group them. Grouping strategies vary from simple clustering  to graph-based algorithms  depending on the embedding characteristics. By performing per-pixel predictions, the local-coherence and position information is well retained. The shortcomings for bottom-up approaches are: 1) heavy reliance on the dense prediction quality, leading to sub-par performance and fragmented/joint masks; 2) limited generalization ability to complex scenes with a large number of classes; 3) requirement for complex post-processing techniques.
In this work, we consider hybridizing top-down and bottom-up approaches. We recognize two important predecessors, FCIS  and YOLACT . They predict instance-level information such as bounding box locations and combine it with per-pixel predictions using cropping (FCIS) and weighted summation (YOLACT), respectively. We argue that these overly simplified assembling designs may not provide a good balance for the representation power of top- and bottom-level features.
Higher-level features correspond to larger receptive field and can better capture overall information about instances such as poses, while lower-level features preserve better location information and can provide finer details. One of the focuses of our work is to investigate ways to better merging these two in fully convolutional instance segmentation. More specifically, we generalize the operations for proposal-based mask combination by enriching the instance-level information and performing more fine-grained position-sensitive mask prediction. We carry out extensive ablation studies to discover the optimal dimensions, resolutions, alignment methods, and feature locations. Concretely, we are able to achieve the followings:
We devise a flexible method for proposal-based instance mask generation called blender, which incorporate rich instance-level information with accurate dense pixel features. In head-to-head comparison, our blender surpasses the merging techniques in YOLACT  and FCIS  by 1.9 and 1.3 points in mAP on the COCO dataset respectively.
We propose a simple architecture, BlendMask, which is closely tied to the state of the art one-stage object detector, FCOS , by adding moldiest computation overhead to the already simple framework.
One obvious advantage of BlendMask is that its inference time does not increase with the number of predictions as conventional two-stage methods do, which makes it more robust in real-time scenarios.
The performance of BlendMask achieves mAP of with the ResNet-50  backbone and mAP with ResNet-101 on the COCO dataset, outperforming Mask R-CNN  in accuracy while being about faster. We set new records for fully convolutional instance segmentation, surpassing TensorMask  by 1.1 points in mask mAP with only half training iterations and inference time.
To our knowledge, BlendMask may be the first algorithm that can outperform Mask R-CNN in both mask AP and inference efficiency.
BlendMask can naturally solve panoptic segmentation without any modification (refer to Section 4.4), as the bottom module of BlendMask can segment ‘things and stuff’ simultaneously.
Compared with Mask R-CNN’s mask head, which is typically of resolution, BlendMask’s the bottom module is able to output masks of much higher resolution, due to its flexibility and the bottom module not being strictly tied to the FPN. Thus BlendMask is able to produce masks with more accurate edges, as shown in Figure 4. For applications such as graphics, this can be very important.
The proposed BlendMask is general and flexible. With minimal modification, we can apply BlendMask to solve other instance-level recognition tasks such as keypoint detection.
2 Related work
Anchor-free object detection Recent advances in object detection unveil the possibilities of removing bounding box anchors , largely simplifying the detection pipeline. This much simpler design improves the box average precision (AP) by 2.7% comparing to its anchor-based counter-part RetinaNet . One possible reason responsible for the improvement is that without the restrictions of predefined anchor shapes, targets are freely matched to prediction features according to their effective receptive field. The hints for us are twofold. First, it is important to map target sizes with proper pyramid levels to fit the effective receptive field for the features. Second, removing anchors enables us to assign heavier duties to the top-level instance prediction module without introducing overall computation overhead. For example, inferring shape and pose information alongside the bounding box detection would take about eight times more computation for anchor-based frameworks than ours. This makes it intractable for anchor based detectors to balance the top vs. bottom workload (i.e., learning instance-aware maps111Attention maps for BlendMask and simple weight scalars for YOLACT. vs. bases). We assume that this might be the reason why YOLACT can only learn one single scalar coefficient for each prototype/basis given an instance when computation complexity is taken into account. Only with the use of anchor-free bounding box detectors, this restriction is removed.
Detect-then-segment instance segmentation The dominant instance segmentation paradigms take the two-stage methodology, first detecting the objects and then predicting the foreground masks on each of the proposals. The success of this framework partially is due to the alignment operation, RoIAlign , which provides local-coherence for the second-stage RoI heads missing in all one-stage top-down approaches. However, two issues exist in two-stage frameworks. For complicated scenarios with many instances, inference time for two-stage methods is proportional to the number of instances. Furthermore, the resolution for the RoI features and resulting mask is limited. We discuss the second issue in detail in Section 4.3.
These problems can be partly solved by replacing a RoI head with a simple crop-and-assemble module. In FCIS, Li et al.  add a bottom module to a detection network, for predicting position-sensitive score maps shared by all instances. This technique was first used in R-FCN  and later improved in MaskLab . Each channel of the score maps corresponds to one crop of evenly partitioned grid tiles of the proposal. Each score map represents the likelihood of the pixel belongs to a object and is at a certain relative position. Naturally, a higher resolution for location crops leads to more accurate predictions, but the computation cost also increases quadratically. Moreover, there are special cases where FCIS representation is not sufficient. When two instances share center positions (or any other relative positions), the score map representation on that crop is ambiguous, it is impossible to tell which instance this crop is describing.
In YOLACT , an improved approach is used. Instead of using position-controlled tiles, a set of mask coefficients are learned alongside the box predictions. Then this set of coefficients guides the linear combination of cropped bottom mask bases to generate the final mask. Comparing to FCIS, the responsibility for predicting instance-level information is assigned to the top-level. We argue that using scalar coefficients to encode the instance information is sub-optimal.
To break through these limitations, we propose a new proposal-based mask generation framework, termed BlendMask. The top- and bottom-level representation workloads are balanced by a blender module. Both levels are guaranteed to describe the instance information within their best capacities. As shown in our experiments in Section 4, our blender module improves the performance of bases combination methods comparing to YOLACT and FCIS by a large margin without increasing computation complexity.
Refining coarse masks with lower-level features BlendMask merges top-level coarse instance information with lower-level fine-granularity. This idea resembles MaskLab  and Instance Mask Projection (IMP) , which concatenates mask predictions with lower layers of backbone features. The differences are clear. Our coarse mask acts like an attention map. The generation is extremely light-weight, without the need of using semantic or positional supervision, and is closely tied to the object generation. As shown in Section 3.4, our lower-level features have clear contextual meanings, even though not explicitly guided by bins or crops. Further, our blender does not require a subnet on top of the merged features as in MaskLab  and IMP , which makes our method more efficient. In parallel to this work recent two single shot instance segmentation methods have shown good performance [25, 24].
3 Our BlendMask
3.1 Overall pipeline
BlendMask consists of a detector network and a mask branch. The mask branch has three parts, a bottom module to predict the score maps, a top layer to predict the instance attentions, and a blender module to merge the scores with attentions. The whole network is illustrated in Figure 2.
Bottom module Similar to other proposal-based fully convolutional methods [16, 2], we add a bottom module predicting score maps which we call bases, . has a shape of , where is the batch size, is the number of bases, is the input size and is the score map output stride. We use the decoder of DeepLabV3+ in our experiments. Other dense prediction modules should also work without much difference. The input for the bottom module could be backbone features like conventional semantic segmentation networks , or the feature pyramids like YOLACT and Panoptic FPN .
Top layer We also append a single convolution layer on each of the detection towers to predict top-level attentions . Unlike the mask coefficients in YOLACT, which for each pyramid with resolution takes the shape of , our
is a tensor at each location with shape, where is the attention resolution. With its 3D structure, our attention map can encode instance-level information, e.g. the coarse shape and pose of the object. is typically smaller222The largest we try is .
than the mask predictions in top-down methods since we only ask for a rough estimate. We predict it with a convolution withoutput channels. Before sending them into the next module, we first apply FCOS  post-process to select the top box predictions and corresponding attentions .
Blender module is the key part of our BlendMask. It combines position-sensitive bases according to the attentions to generate the final prediction. We discuss this module in detail in the next section.
3.2 Blender module
The inputs of the blender module are bottom-level bases , the selected top-level attentions and bounding box proposals . First we use RoIPooler in Mask R-CNN  to crop bases with each proposal and then resize the region to a fixed size feature map .
More specifically, we use sampling ratio for RoIAlign, i.e. one bin for each sampling point. The performance of using nearest and bilinear poolers are compared in Table 6. During training, we simply use ground truth boxes as the proposals. During inference, we use FCOS prediction results.
Our attention size is smaller than
. We interpolatefrom to , into the shapes of .
Then is normalize with softmax function along the dimension to make it a set of score maps .
Then we apply element-wise product between each entity , of the regions and scores , and sum along the
dimension to get our mask logit:
where is the index of the basis. We visualize the mask blending process with in Figure 1.
3.3 Configurations and baselines
We consider the following configurable hyper-parameters for BlendMask:
, the bottom-level RoI resolution,
, the top-level prediction resolution,
, the number of bases,
bottom module input features, it can either be features from the backbone or the FPN,
sampling method for bottom bases, nearest-neighbour or bilinear pooling,
interpolation method for top-level attentions, nearest neighbour or bilinear upsampling.
We represent our models with abbreviation . For example, represents bottom-level region resolution of , number of bases and top-level instance attentions. By default, we use backbone features C3 and C5 to keep aligned with DeepLabv3+ . Nearest neighbour interpolation is used in top-level interpolation, for a fair comparison with FCIS . Bilinear sampling is used in the bottom level, consistent with RoIAlign .
3.4 Semantics encoded in learned bases and attentions
By examining the generated bases and attentions on , we observe this pattern. On its bases, BlendMask encodes two types of local information, 1) whether the pixel is on an object (semantic masks), 2) whether the pixel is on certain part of the object (position-sensitive features).
The complete bases and attentions projected onto the original image are illustrated in Figure 3. The first two bases (red and blue) detects points on the upper-right and bottom-left parts of the objects. The third (yellow) base activates on points more likely to be on an object. The fourth (green) base only activates on the borders of objects. Position-sensitive features help us separate overlapping instances, which enables BlendMask to represent all instances more efficiently than YOLACT . The positive semantic mask makes our final prediction smoother than FCIS  and the negative one can further suppress out-of-instance activations. We compare our blender with YOLACT and FCIS counterparts in Table 1. BlendMask can learn more accurate features than YOLACT and FCIS with much fewer number of bases (4 vs. 32 vs. 49, see Section 4.2).
Our experiments are reported on the MSCOCO 2017 instance segmentation datatset . It contains 123K images with 80-class instance labels. Our models are trained on the split (115K images) and the ablation study is carried out on the split (5K images). Final results are on -
. The evaluation metrics are COCO mask average precision (AP), AP at IoU 0.5 (AP), 0.75 (AP) and AP for objects at different sizes AP, AP, and AP.
Unless specified, ImageNet pre-trained ResNet-50 is used as our backbone network. DeepLabv3+  with channel width is used as our bottom module. For ablation study, all the networks are trained with the schedule of FCOS 
, i.e., 90K iterations, batch size 16 on 4 GPUs, and base learning rate 0.01 with constant warm-up of 1k iterations. The learning rate is reduced by a factor of 10 at iteration 60K and 80K. Input images are resized to have shorter side 800 and longer side at maximum 1333. All hyperparameters are set to be the same with FCOS.
Testing details The unit for inference time is ‘ms’ in all our tables. For the ablation experiments, performance and time of our models are measured with one image per batch on one 1080Ti GPU.
4.1 Ablation experiments
We investigate the effectiveness of our blender module by carrying out ablation experiments on the configurable hyperparameters in Section 3.3.
Merging methods: Blender vs. YOLACT vs. FCIS Similar to our method, YOLACT  and FCIS  both merge proposal-based bottom regions to create mask prediction. YOLACT simply performs a weighted sum of the channels of the bottom regions; FCIS assembles crops of position-sensitive masks without modifications. Our blender can be regarded as a generalization where both YOLACT and FCIS merging are special cases: The blender with top-level resolution degenerates to YOLACT; and FCIS is the case where we use fixed one-hot blending attentions and nearest neighbour top-level interpolation.
Results of these variations are shown in Table 1. Our blender surpasses the other alternatives by a large margin. We assume the reason is that other methods lack instance-aware guidance on the top. By contrast, our blender has a fine-grained top-level attention map, as illustrated in Figure 3.
Top and bottom resolutions: We measure the performances of our model with different top- and bottom-level resolutions, trying bottom pooler resolution being and , with ratio from to . As shown in Table 2, by increasing the attention resolution, we can incorporate more detailed instance-level information while keeping the running time roughly the same. Notice that the gain slows down at higher resolutions revealing limit of detailed information on the top-level. So we don’t include larger top settings with ratio smaller than .
Different from two-stage approaches, increasing the bottom-level bases pooling resolution does not introduce much computation overhead. Increasing it from to only increases the inference time within 0.2ms while mask AP increases by 1 point. In further ablation experiment, we set and for our baseline model if not specified.
Number of bases: YOLACT  uses bases concerning the inference time. With our blender, the number of bases can be further reduced, to even just one. We report our models with number of bases varying from to . Different from normal blender, the one-basis version uses sigmoid activation on both the base and the attention map. Results are shown in Table 3. Since instance-level information is better represented with the top-level attentions, we only need bases to get the optimal accuracy. is adopted by all subsequent experiments.
Bottom feature locations: backbone vs. FPN We compare our bottom module feature sampling locations. By using FPN features, we can improve the performance while reducing the running time (see Table 4). In later experiments, if not specified, we use P3 and P5 of FPN as our bottom module input.
|+deform convs ||72||✓||-||40.4||62.7||43.3||17.6||43.3||58.9|
|+deform convs (interval )||60||✓||116.0||41.3||63.1||44.6||22.7||44.1||54.5|
Interpolation method: nearest vs. bilinear In Mask R-CNN , RoIAlign plays a crucial role in aligning the pooled features to keep local-coherence. We investigate the effectiveness of bilinear interpolation for bottom RoI sampling and top-level attention re-scaling. As shown in Table 5, changing top interpolation from nearest to bilinear yields a marginal improvement of 0.2 AP.
The results of bottom sampling with RoIPool  (nearest) and RoIAlign  (bilinear) are shown in Table 6. For both resolutions, the aligned bilinear sampling could improve the performance by almost 2AP. Using aligned features for the bottom-level is more crucial, since it is where the detailed positions are predicted. Bilinear top and bottom interpolation are adopted for our final models.
Other improvements: We experiment on other tricks to improve the performance. First we add auxiliary semantic segmentation supervision on P3 similar to YOLACT . Then we increase the width of our bottom module from to . Finally, we reduce the bases output stride from to , to produce higher-quality bases. We achieve this by using P2 and P5 as the bottom module input. Table 7 shows the results. By adding semantic loss, detection and segmentation results are both improved. This is an interesting effect since the instance segmentation task itself does not improve the box AP. Although all tricks contribute to the improvements, we decide to not use larger basis resolution because it slows down the model by 10ms per image.
We also implement the protonet module in YOLACT  for comparison. We include a P3 version and an FPN version. The P3 version is identical to the one used in YOLACT. For the FPN version, we first change the channel width of P3, P4, and P5 to 128 with a convolution. Then upsample all features to s/8 and sum them up. Following are the same as P2 version except that we reduce convolution layers by one. Auxiliary semantic loss is applied to both versions. As shown in Table 7, changing the bottom module from DeepLabv3+ to protonet does not modify the speed and performance significantly.
4.2 Main result
Quantitative results We compare BlendMask with Mask R-CNN  and TensorMask  on the COCO - dataset333To make fair comparison with TensorMask, the code base that we use for main result is . Recently released fixed several issues of ( and ) in the previous repository and the performance is further improved.. We use with bilinear top interpolation, the DeepLabV3+ decoder with channel width 256 and P3, P5 input. Since our ablation models are heavily under-fitted, we increase the training iterations to 270K ( schedule), tuning learning rate down at 180K and 240K. Following Chen et al.’s strategy , we use multi-scale training with shorter side randomly sampled from . As shown in Table 8, our BlendMask outperforms both the modified Mask R-CNN with deeper FPN and TensorMask using only half of their training iterations.
BlendMask is also more efficient. Measured on a V100 GPU, the best R-101 BlendMask runs at 0.07s/im, vs. TensorMask’s 0.38s/im, vs. Mask R-CNN’s 0.09s/im . Furthermore, a typical running time of our blender module is merely 0.6ms, which makes the additional time for complex scenes nearly negligible On the contrary, for two-stage Mask R-CNN with more expensive head computation, the inference time increases by a lot if the number of predicted instances grows.
Real-time setting We design a compact version of our model, BlendMask-RT, to compare with YOLACT , a real-time instance segmentation method: i) the number of convolution layers in the prediction head is reduced to three, ii) and we merge the classification tower and box tower into one by sharing their features. We use Proto-FPN with four convolution layers with width 128 as the bottom module. The top FPN output P7 is removed because it has little effect on the detecting smaller objects. We train both BlendMask-RT and Mask R-CNN with the schedule, with shorter side randomly sampled from .
There are still two differences in the implementation comparing to YOLACT. YOLACT resizes all images to square, changing the aspect ratios of inputs. Also, a paralleled NMS algorithm called Fast NMS is used in YOLACT. We do not adopt these two configurations because they are not conventionally used in instance segmentation researches. In YOLACT, a speedup of 12ms is reported by using Fast NMS. We instead use the Batched NMS in , which could be slower than Fast NMS but does not sacrifice the accuracy. Results in Table 9 shows that BlendMask-RT is 7ms faster and 3.3 AP higher than YOLACT-700. Making our model also competitive under the real-time settings.
Qualitative results We compare our model with the best available official YOLACT and Mask R-CNN models with ResNet-101 backbone. Masks are illustrated in Figure 4. Our model yields higher quality masks than Mask R-CNN. The first reason is that we predicts masks while Mask R-CNN uses masks. Also our segmentation module mostly utilizes high resolution features that preserve the original aspect-ratio, where Mask R-CNN also uses features.
Note that YOLACT has difficulties discriminating instances of the same class close to each other. BlendMask can avoid this typical leakage. This is because its top module provides more detailed instance-level information, guiding the bases to capture position-sensitive information and suppressing the outside regions.
Comparison with Mask R-CNN Similar to Mask R-CNN, we use RoIPooler to locate instances and extract features. We reduce the running time by moving the computation of R-CNN heads before the RoI sampling to generate position-sensitive feature maps. Repeated mask representation and computation for overlapping proposals are avoided. We further simplify the global map representation by replacing the hard alignment in R-FCN  and FCIS  with our attention guided blender, which needs ten times less channels for the same resolution.
Another advantage of BlendMask is that it can produce higher quality masks, since our output resolution is not restricted by the top-level sampling. Increasing the RoIPooler resolution of Mask R-CNN will introduce the following problem. The head computation increases quadratically with respect to the RoI size. Larger RoIs requires deeper head structures. Different from dense pixel predictions, RoI foreground predictor has to be aware of whole instance-level information to distinguish foreground from other overlapping instances. Thus, the larger the feature sizes are, the deeper sub-networks is needed.
Furthermore, it is not very friendly to real-time applications that the inference time of Mask R-CNN is proportional to the number of detections. By contrast, our blender module is very efficient (0.6ms on 1080 Ti). The additional inference time required after increasing the number of detections can be neglected.
Our blender module is very flexible. Because our top-level instance attention prediction is just a single convolution layer, it can be an almost free add-on to most modern object detectors. With its accurate instance prediction, it can also be used to refine two-stage instance predictions.
4.4 Panoptic Segmentation
We use the semantic segmentation branch of Panoptic-FPN  to extend BlendMask to the panoptic segmentation task. We use annotations of COCO 2018 panoptic segmentaiton task. All models are trained on subset and tested on . We train our model with the default FCOS  schedule with scale jitter (shorter image side in . To combine instance and semantic results, we use the same strategy as in Panoptic-FPN, with instance confidence threshhold 0.2 and overlap threshhold 0.4.
Results are reported in Table 10. Our model is consistently better than its Mask R-CNN counterpart, Panoptic-FPN. We assume there are three reasons. First, our instance segmentation is more accurate, this helps with both thing and stuff panoptic quality because instance masks are overlaid on top of semantic masks. Second, our pixel-level instance prediction is also generated from a global feature map, which has the same scale as the semantic prediction, thus the two results are more consistent. Last but not least, since the our bottom module shares structure with the semantic segmentation branch, it is easier for the network to share features during the closely related multi-task learning.
4.5 More Qualitative Results
We visualize qualitative results of Mask R-CNN and BlendMask on the validation set in Fig. 5. Four sets of images are listed in rows. Within each set, the top row is the Mask R-CNN results and the bottom is BlendMask. Both models are based on the newly released with use R101-FPN backbone. Both are trained with the schedule. The Mask R-CNN model achieves 38.6% AP and ours 39.5% AP.
Since this version of Mask R-CNN is a very strong baseline, and both models achieve very high accuracy, it is very difficult to tell the differences. To demonstrate our advantage, we select some samples where Mask R-CNN has trouble dealing with. Those cases include:
Large objects with complex shapes (Horse ears, human poses). Mask R-CNN fails to provide sharp borders.
Objects in separated parts (tennis players occluded by nets, trains divided by poles). Mask R-CNN tends to include occlusions as false positive or segment targets into separate objects.
Overlapping objects (riders, crowds, drivers). Mask R-CNN gets uncertain on the borders and leaves larger false negative regions. Sometimes, it assigns parts to the wrong objects, such as the last example in the first row.
Our BlendMask performs better on these cases. 1) Generally, BlendMask utilizes features with higher resolution. Even for the large objects, we use stride-8 features. Thus details are better preserved. 2) As shown in previous illustrations, our bottom module acts as a class agnostic instance segmenter which is very sensitive to borders. 3) Sharing features with the bounding box regressor, our top module is very good at recognizing individual instances. It can generate attentions with flexible shapes to merge the fine-grained segments of bottom module outputs.
We have devised a novel blender module for instance-level dense prediction tasks which uses both high-level instance and low-level semantic information. It is efficient and easy to integrate with different main-stream detection networks.
Our framework BlendMask outperforms the carefully-engineered Mask R-CNN without bells and whistles while being faster. Furthermore, the real-time version BlendMask-RT achieves 34.2% mAP at 25 FPS evaluated on a single 1080Ti GPU card. We believe that our BlendMask is capable of serving as an alternative to Mask R-CNN  for many other instance-level recognition tasks.
The authors would like to thank Huawei Technologies for the donation of GPU cloud computing resources.
-  Anurag Arnab and Philip H. S. Torr. Bottom-up instance segmentation using deep higher-order CRFs. In Proc. British Conf. Machine Vis., 2016.
-  Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. YOLACT: real-time instance segmentation. Proc. Int. Conf. Comp. Vis., abs/1904.02689, 2019.
-  Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. YOLACT++: Better real-time instance segmentation. arXiv Comput. Res. Repository, abs/1912.06218, 2019.
-  Bert De Brabandere, Davy Neven, and Luc Van Gool. Semantic instance segmentation with a discriminative loss function. arXiv Comput. Res. Repository, abs/1708.02551, 2017.
-  Liang-Chieh Chen, Alexander Hermans, George Papandreou, Florian Schroff, Peng Wang, and Hartwig Adam. Masklab: Instance segmentation by refining object detection with semantic and direction features. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 4013–4022, 2018.
-  Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. Eur. Conf. Comp. Vis., pages 833–851, 2018.
-  Xinlei Chen, Ross B. Girshick, Kaiming He, and Piotr Dollár. Tensormask: A foundation for dense object segmentation. Proc. Int. Conf. Comp. Vis., abs/1903.12174, 2019.
-  Jifeng Dai, Kaiming He, Yi Li, Shaoqing Ren, and Jian Sun. Instance-sensitive fully convolutional networks. In Proc. Eur. Conf. Comp. Vis., pages 534–549, 2016.
-  Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-FCN: object detection via region-based fully convolutional networks. In Proc. Adv. Neural Inf. Process. Syst., pages 379–387, 2016.
-  Cheng-Yang Fu, Tamara L. Berg, and Alexander C. Berg. IMP: instance mask projection for high accuracy semantic segmentation of things. arXiv Comput. Res. Repository, abs/1906.06597, 2019.
-  Ross B. Girshick. Fast R-CNN. In Proc. Int. Conf. Comp. Vis., pages 1440–1448, 2015.
-  Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. In Proc. Int. Conf. Comp. Vis., pages 2980–2988, 2017.
-  Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 770–778, 2016.
-  Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Proc. Eur. Conf. Comp. Vis., pages 630–645, 2016.
-  Alexander Kirillov, Ross B. Girshick, Kaiming He, and Piotr Dollár. Panoptic feature pyramid networks. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 6399–6408, 2019.
-  Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. Fully convolutional instance-aware semantic segmentation. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 4438–4446, 2017.
-  Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proc. Int. Conf. Comp. Vis., pages 2999–3007, 2017.
-  Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In Proc. Eur. Conf. Comp. Vis., pages 740–755, 2014.
-  Yiding Liu, Siyu Yang, Bin Li, Wengang Zhou, Jizheng Xu, Houqiang Li, and Yan Lu. Affinity derivation and graph merge for instance segmentation. In Proc. Eur. Conf. Comp. Vis., pages 708–724, 2018.
-  Davy Neven, Bert De Brabandere, Marc Proesmans, and Luc Van Gool. Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 8837–8845, 2019.
-  Pedro H. O. Pinheiro, Ronan Collobert, and Piotr Dollár. Learning to segment object candidates. In Proc. Adv. Neural Inf. Process. Syst., pages 1990–1998, 2015.
-  Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with region proposal networks. In Proc. Adv. Neural Inf. Process. Syst., pages 91–99, 2015.
-  Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: fully convolutional one-stage object detection. Proc. Int. Conf. Comp. Vis., abs/1904.01355, 2019.
-  Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li. SOLO: Segmenting objects by locations. arXiv Comput. Res. Repository, 2019. arxiv.org/abs/1912.04488.
-  Enze Xie, Peize Sun, Xiaoge Song, Wenhai Wang, Xuebo Liu, Ding Liang, Chunhua Shen, and Ping Luo. PolarMask: Single shot instance segmentation with polar representation. arXiv Comput. Res. Repository, 2019. arxiv.org/abs/1909.13226.