DeepAI AI Chat
Log In Sign Up

ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

by   Angela Dai, et al.

We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels. The key contribution of our method is its ability to handle large scenes with varying spatial extent, managing the cubic growth in data size as scene size increases. To this end, we devise a fully-convolutional generative 3D CNN model whose filter kernels are invariant to the overall scene size. The model can be trained on scene subvolumes but deployed on arbitrarily large scenes at test time. In addition, we propose a coarse-to-fine inference strategy in order to produce high-resolution output while also leveraging large input context sizes. In an extensive series of experiments, we carefully evaluate different model design choices, considering both deterministic and probabilistic models for completion and semantic inference. Our results show that we outperform other methods not only in the size of the environments handled and processing efficiency, but also with regard to completion quality and semantic segmentation performance by a significant margin.


page 1

page 3

page 7

page 8

page 11

page 13

page 14

page 15


3D-SIC: 3D Semantic Instance Completion for RGB-D Scans

This paper focuses on the task of semantic instance completion: from an ...

CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion

Semantic scene completion (SSC) aims to complete a partial 3D scene and ...

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

We present a novel approach that converts partial and noisy RGB-D scans ...

In Depth Bayesian Semantic Scene Completion

This work studies Semantic Scene Completion which aims to predict a 3D s...

DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference

Semantic segmentation for scene understanding is nowadays widely demande...

Unsupervised Continual Semantic Adaptation through Neural Rendering

An increasing amount of applications rely on data-driven models that are...

Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective

We revisit Semantic Scene Completion (SSC), a useful task to predict the...