Unsupervised 3D Shape Completion through GAN Inversion

04/27/2021
by   Junzhe Zhang, et al.
12

Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best reconstructs the given partial input. In this way, ShapeInversion no longer needs paired training data, and is capable of incorporating the rich prior captured in a well-trained generative model. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data. It also demonstrates remarkable generalization ability, giving robust results for real-world scans and partial inputs of various forms and incompleteness levels. Importantly, ShapeInversion naturally enables a series of additional abilities thanks to the involvement of a pre-trained GAN, such as producing multiple valid complete shapes for an ambiguous partial input, as well as shape manipulation and interpolation.

READ FULL TEXT

page 2

page 10

page 11

page 12

page 13

page 14

research
03/17/2020

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Several deep learning methods have been proposed for completing partial ...
research
06/30/2021

Shape Completion via IMLE

Shape completion is the problem of completing partial input shapes such ...
research
03/29/2019

Unpaired Point Cloud Completion on Real Scans using Adversarial Training

As 3D scanning solutions become increasingly popular, several deep learn...
research
09/07/2022

3D Textured Shape Recovery with Learned Geometric Priors

3D textured shape recovery from partial scans is crucial for many real-w...
research
08/11/2023

U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds

In this paper, we propose U-RED, an Unsupervised shape REtrieval and Def...
research
03/24/2023

Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken Objects and Their Complete Counterparts

Automated shape repair approaches currently lack access to datasets that...
research
04/19/2021

Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective

We present a novel approach for completing and reconstructing 3D shapes ...

Please sign up or login with your details

Forgot password? Click here to reset