Probabilistic Implicit Scene Completion

04/04/2022
by   Dongsu Zhang, et al.
0

We propose a probabilistic shape completion method extended to the continuous geometry of large-scale 3D scenes. Real-world scans of 3D scenes suffer from a considerable amount of missing data cluttered with unsegmented objects. The problem of shape completion is inherently ill-posed, and high-quality result requires scalable solutions that consider multiple possible outcomes. We employ the Generative Cellular Automata that learns the multi-modal distribution and transform the formulation to process large-scale continuous geometry. The local continuous shape is incrementally generated as a sparse voxel embedding, which contains the latent code for each occupied cell. We formally derive that our training objective for the sparse voxel embedding maximizes the variational lower bound of the complete shape distribution and therefore our progressive generation constitutes a valid generative model. Experiments show that our model successfully generates diverse plausible scenes faithful to the input, especially when the input suffers from a significant amount of missing data. We also demonstrate that our approach outperforms deterministic models even in less ambiguous cases with a small amount of missing data, which infers that probabilistic formulation is crucial for high-quality geometry completion on input scans exhibiting any levels of completeness.

READ FULL TEXT

page 7

page 24

page 26

page 27

page 28

page 29

page 30

page 31

research
04/08/2021

3D Shape Generation and Completion through Point-Voxel Diffusion

We propose a novel approach for probabilistic generative modeling of 3D ...
research
03/06/2021

Learning to Generate 3D Shapes with Generative Cellular Automata

We present a probabilistic 3D generative model, named Generative Cellula...
research
06/27/2019

Variational Shape Completion for Virtual Planning of Jaw Reconstructive Surgery

The premorbid geometry of the mandible is of significant relevance in ja...
research
06/27/2019

Variational Mandible Shape Completion for Virtual Surgical Planning

The premorbid geometry of the mandible is of significant relevance in ja...
research
03/17/2020

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Several deep learning methods have been proposed for completing partial ...
research
06/28/2023

DiffComplete: Diffusion-based Generative 3D Shape Completion

We introduce a new diffusion-based approach for shape completion on 3D r...
research
07/29/2018

PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction

We propose a novel 3D spatial representation for data fusion and scene r...

Please sign up or login with your details

Forgot password? Click here to reset