Learning to Infer 3D Shape Programs with Differentiable Renderer

by   Yichao Liang, et al.

Given everyday artifacts, such as tables and chairs, humans recognize high-level regularities within them, such as the symmetries of a table, the repetition of its legs, while possessing low-level priors of their geometries, e.g., surfaces are smooth and edges are sharp. This kind of knowledge constitutes an important part of human perceptual understanding and reasoning. Representations of and how to reason in such knowledge, and the acquisition thereof, are still open questions in artificial intelligence (AI) and cognitive science. Building on the previous proposal of the 3D shape programs representation alone with the accompanying neural generator and executor from <cit.>, we propose an analytical yet differentiable executor that is more faithful and controllable in interpreting shape programs (particularly in extrapolation) and more sample efficient (requires no training). These facilitate the generator's learning when ground truth programs are not available, and should be especially useful when new shape-program components are enrolled either by human designers or – in the context of library learning – algorithms themselves. Preliminary experiments on using it for adaptation illustrate the aforesaid advantages of the proposed module, encouraging similar methods being explored in building machines that learn to reason with the kind of knowledge described above, and even learn this knowledge itself.


page 3

page 4

page 7


Learning to Infer and Execute 3D Shape Programs

Human perception of 3D shapes goes beyond reconstructing them as a set o...

Cognitive Interpretation of Everyday Activities: Toward Perceptual Narrative Based Visuo-Spatial Scene Interpretation

We position a narrative-centred computational model for high-level knowl...

Differentiable programming: Generalization, characterization and limitations of deep learning

In the past years, deep learning models have been successfully applied i...

Incremental and Iterative Learning of Answer Set Programs from Mutually Distinct Examples

Over these years the Artificial Intelligence (AI) community has produced...

Learning to Infer Shape Programs Using Latent Execution Self Training

Inferring programs which generate 2D and 3D shapes is important for reve...

An Approach for Solving Tasks on the Abstract Reasoning Corpus

The Abstract Reasoning Corpus (ARC) is an intelligence tests for measuri...

Null Dynamical State Models of Human Cognitive Dysfunction

The hard problem in artificial intelligence asks how the shuffling of sy...

Please sign up or login with your details

Forgot password? Click here to reset