Occlusion-Aware Instance Segmentation via BiLayer Network Architectures

08/08/2022
by   Lei Ke, et al.
21

Segmenting highly-overlapping image objects is challenging, because there is typically no distinction between real object contours and occlusion boundaries on images. Unlike previous instance segmentation methods, we model image formation as a composition of two overlapping layers, and propose Bilayer Convolutional Network (BCNet), where the top layer detects occluding objects (occluders) and the bottom layer infers partially occluded instances (occludees). The explicit modeling of occlusion relationship with bilayer structure naturally decouples the boundaries of both the occluding and occluded instances, and considers the interaction between them during mask regression. We investigate the efficacy of bilayer structure using two popular convolutional network designs, namely, Fully Convolutional Network (FCN) and Graph Convolutional Network (GCN). Further, we formulate bilayer decoupling using the vision transformer (ViT), by representing instances in the image as separate learnable occluder and occludee queries. Large and consistent improvements using one/two-stage and query-based object detectors with various backbones and network layer choices validate the generalization ability of bilayer decoupling, as shown by extensive experiments on image instance segmentation benchmarks (COCO, KINS, COCOA) and video instance segmentation benchmarks (YTVIS, OVIS, BDD100K MOTS), especially for heavy occlusion cases. Code and data are available at https://github.com/lkeab/BCNet.

READ FULL TEXT

page 2

page 5

page 7

page 8

page 9

page 10

page 11

page 12

research
03/23/2021

Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers

Segmenting highly-overlapping objects is challenging, because typically ...
research
02/14/2020

Layered Embeddings for Amodal Instance Segmentation

The proposed method extends upon the representational output of semantic...
research
03/31/2020

BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation

Panoptic segmentation aims to perform instance segmentation for foregrou...
research
10/07/2022

Humans need not label more humans: Occlusion Copy Paste for Occluded Human Instance Segmentation

Modern object detection and instance segmentation networks stumble when ...
research
05/22/2022

Human Instance Matting via Mutual Guidance and Multi-Instance Refinement

This paper introduces a new matting task called human instance matting (...
research
07/22/2020

Deep Variational Instance Segmentation

Instance Segmentation, which seeks to obtain both class and instance lab...
research
10/07/2022

Instance Segmentation of Dense and Overlapping Objects via Layering

Instance segmentation aims to delineate each individual object of intere...

Please sign up or login with your details

Forgot password? Click here to reset