Building 3D Generative Models from Minimal Data

03/04/2022
by   Skylar Sutherland, et al.
20

We propose a method for constructing generative models of 3D objects from a single 3D mesh and improving them through unsupervised low-shot learning from 2D images. Our method produces a 3D morphable model that represents shape and albedo in terms of Gaussian processes. Whereas previous approaches have typically built 3D morphable models from multiple high-quality 3D scans through principal component analysis, we build 3D morphable models from a single scan or template. As we demonstrate in the face domain, these models can be used to infer 3D reconstructions from 2D data (inverse graphics) or 3D data (registration). Specifically, we show that our approach can be used to perform face recognition using only a single 3D template (one scan total, not one per person). We extend our model to a preliminary unsupervised learning framework that enables the learning of the distribution of 3D faces using one 3D template and a small number of 2D images. This approach could also provide a model for the origins of face perception in human infants, who appear to start with an innate face template and subsequently develop a flexible system for perceiving the 3D structure of any novel face from experience with only 2D images of a relatively small number of familiar faces.

READ FULL TEXT

page 22

page 34

page 35

page 37

page 38

page 39

page 40

page 42

research
11/24/2020

Building 3D Morphable Models from a Single Scan

We propose a method for constructing generative models of 3D objects fro...
research
07/06/2016

Pooling Faces: Template based Face Recognition with Pooled Face Images

We propose a novel approach to template based face recognition. Our dual...
research
03/06/2022

Face Recognition using a variation of principle component analysis technique

Face recognition systems are built on computer programs that analyze ima...
research
12/20/2013

Learning Generative Models with Visual Attention

Attention has long been proposed by psychologists as important for effec...
research
03/20/2023

Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images

The function of constructing the hierarchy of objects is important to th...
research
06/18/2018

Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance

In this work we introduce Deforming Autoencoders, a generative model for...
research
08/14/2019

MemeFaceGenerator: Adversarial Synthesis of Chinese Meme-face from Natural Sentences

Chinese meme-face is a special kind of internet subculture widely spread...

Please sign up or login with your details

Forgot password? Click here to reset