Unsupervised 3D Shape Completion through GAN Inversion

04/27/2021
by   Junzhe Zhang, et al.
12

Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best reconstructs the given partial input. In this way, ShapeInversion no longer needs paired training data, and is capable of incorporating the rich prior captured in a well-trained generative model. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data. It also demonstrates remarkable generalization ability, giving robust results for real-world scans and partial inputs of various forms and incompleteness levels. Importantly, ShapeInversion naturally enables a series of additional abilities thanks to the involvement of a pre-trained GAN, such as producing multiple valid complete shapes for an ambiguous partial input, as well as shape manipulation and interpolation.

READ FULL TEXT

page 2

page 10

page 11

page 12

page 13

page 14

03/17/2020

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Several deep learning methods have been proposed for completing partial ...
06/30/2021

Shape Completion via IMLE

Shape completion is the problem of completing partial input shapes such ...
03/29/2019

Unpaired Point Cloud Completion on Real Scans using Adversarial Training

As 3D scanning solutions become increasingly popular, several deep learn...
09/07/2022

3D Textured Shape Recovery with Learned Geometric Priors

3D textured shape recovery from partial scans is crucial for many real-w...
07/24/2022

PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation

This paper introduces a data-driven shape completion approach that focus...
04/10/2020

Hamiltonian Dynamics for Real-World Shape Interpolation

We revisit the classical problem of 3D shape interpolation and propose a...
03/14/2021

Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Coding

In this paper, we present a novel unpaired point cloud completion networ...

Code Repositories

shape-inversion

[CVPR 2021] Unsupervised 3D Shape Completion through GAN Inversion


view repo