Learning to Infer and Execute 3D Shape Programs

01/09/2019
by   Yonglong Tian, et al.
6

Human perception of 3D shapes goes beyond reconstructing them as a set of points or a composition of geometric primitives: we also effortlessly understand higher-level shape structure such as the repetition and reflective symmetry of object parts. In contrast, recent advances in 3D shape sensing focus more on low-level geometry but less on these higher-level relationships. In this paper, we propose 3D shape programs, integrating bottom-up recognition systems with top-down, symbolic program structure to capture both low-level geometry and high-level structural priors for 3D shapes. Because there are no annotations of shape programs for real shapes, we develop neural modules that not only learn to infer 3D shape programs from raw, unannotated shapes, but also to execute these programs for shape reconstruction. After initial bootstrapping, our end-to-end differentiable model learns 3D shape programs by reconstructing shapes in a self-supervised manner. Experiments demonstrate that our model accurately infers and executes 3D shape programs for highly complex shapes from various categories. It can also be integrated with an image-to-shape module to infer 3D shape programs directly from an RGB image, leading to 3D shape reconstructions that are both more accurate and more physically plausible.

READ FULL TEXT

page 4

page 9

research
11/25/2020

Learning to Infer Shape Programs Using Latent Execution Self Training

Inferring programs which generate 2D and 3D shapes is important for reve...
research
12/19/2022

GeoCode: Interpretable Shape Programs

Mapping high-fidelity 3D geometry to a representation that allows for in...
research
06/25/2022

Learning to Infer 3D Shape Programs with Differentiable Renderer

Given everyday artifacts, such as tables and chairs, humans recognize hi...
research
04/29/2018

Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers

In this paper, we develop novel, efficient 2D encodings for 3D geometry,...
research
03/03/2023

Unsupervised 3D Shape Reconstruction by Part Retrieval and Assembly

Representing a 3D shape with a set of primitives can aid perception of s...
research
09/17/2020

ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis

Manually authoring 3D shapes is difficult and time consuming; generative...
research
12/22/2019

Neural Shape Parsers for Constructive Solid Geometry

Constructive Solid Geometry (CSG) is a geometric modeling technique that...

Please sign up or login with your details

Forgot password? Click here to reset