3D Instances as 1D Kernels

07/15/2022
by   Yizheng Wu, et al.
8

We introduce a 3D instance representation, termed instance kernels, where instances are represented by one-dimensional vectors that encode the semantic, positional, and shape information of 3D instances. We show that instance kernels enable easy mask inference by simply scanning kernels over the entire scenes, avoiding the heavy reliance on proposals or heuristic clustering algorithms in standard 3D instance segmentation pipelines. The idea of instance kernel is inspired by recent success of dynamic convolutions in 2D/3D instance segmentation. However, we find it non-trivial to represent 3D instances due to the disordered and unstructured nature of point cloud data, e.g., poor instance localization can significantly degrade instance representation. To remedy this, we construct a novel 3D instance encoding paradigm. First, potential instance centroids are localized as candidates. Then, a candidate merging scheme is devised to simultaneously aggregate duplicated candidates and collect context around the merged centroids to form the instance kernels. Once instance kernels are available, instance masks can be reconstructed via dynamic convolutions whose weights are conditioned on instance kernels. The whole pipeline is instantiated with a dynamic kernel network (DKNet). Results show that DKNet outperforms the state of the arts on both ScanNetV2 and S3DIS datasets with better instance localization. Code is available: https://github.com/W1zheng/DKNet.

READ FULL TEXT

page 2

page 12

page 20

page 21

page 22

research
03/12/2020

Conditional Convolutions for Instance Segmentation

We propose a simple yet effective instance segmentation framework, terme...
research
02/05/2021

Instance and Panoptic Segmentation Using Conditional Convolutions

We propose a simple yet effective framework for instance and panoptic se...
research
08/17/2021

Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks

Instance segmentation in 3D scenes is fundamental in many applications o...
research
11/26/2020

DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution

Previous top-performing approaches for point cloud instance segmentation...
research
02/02/2023

Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt

Recently, inspired by DETR variants, query-based end-to-end instance seg...
research
08/05/2021

Hierarchical Aggregation for 3D Instance Segmentation

Instance segmentation on point clouds is a fundamental task in 3D scene ...
research
07/04/2022

Beyond mAP: Re-evaluating and Improving Performance in Instance Segmentation with Semantic Sorting and Contrastive Flow

Top-down instance segmentation methods improve mAP by hedging bets on lo...

Please sign up or login with your details

Forgot password? Click here to reset