Learning to Infer Implicit Surfaces without 3D Supervision

11/02/2019
by   Shichen Liu, et al.
12

Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. We demonstrate the effectiveness of our framework on the task of single-view image-based 3D shape digitization and show how we outperform state-of-the-art techniques both quantitatively and qualitatively.

READ FULL TEXT

page 7

page 8

research
07/20/2020

Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling

We propose a novel neural architecture for representing 3D surfaces, whi...
research
10/26/2020

Neural Unsigned Distance Fields for Implicit Function Learning

In this work we target a learnable output representation that allows con...
research
06/20/2021

DeepMesh: Differentiable Iso-Surface Extraction

Geometric Deep Learning has recently made striking progress with the adv...
research
01/01/2022

SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators

Recent advances in deep generative models have led to immense progress i...
research
02/05/2022

Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis

Neural implicit representations, which encode a surface as the level set...
research
03/16/2022

Occlusion Fields: An Implicit Representation for Non-Line-of-Sight Surface Reconstruction

Non-line-of-sight reconstruction (NLoS) is a novel indirect imaging moda...
research
03/23/2018

Learning Shape-from-Shading for Deformable Surfaces

Recent years have seen the development of mature solutions for reconstru...

Please sign up or login with your details

Forgot password? Click here to reset