JNR: Joint-based Neural Rig Representation for Compact 3D Face Modeling

07/14/2020
by   Noranart Vesdapunt, et al.
0

In this paper, we introduce a novel approach to learn a 3D face model using a joint-based face rig and a neural skinning network. Thanks to the joint-based representation, our model enjoys some significant advantages over prior blendshape-based models. First, it is very compact such that we are orders of magnitude smaller while still keeping strong modeling capacity. Second, because each joint has its semantic meaning, interactive facial geometry editing is made easier and more intuitive. Third, through skinning, our model supports adding mouth interior and eyes, as well as accessories (hair, eye glasses, etc.) in a simpler, more accurate and principled way. We argue that because the human face is highly structured and topologically consistent, it does not need to be learned entirely from data. Instead we can leverage prior knowledge in the form of a human-designed 3D face rig to reduce the data dependency, and learn a compact yet strong face model from only a small dataset (less than one hundred 3D scans). To further improve the modeling capacity, we train a skinning weight generator through adversarial learning. Experiments on fitting high-quality 3D scans (both neutral and expressive), noisy depth images, and RGB images demonstrate that its modeling capacity is on-par with state-of-the-art face models, such as FLAME and Facewarehouse, even though the model is 10 to 20 times smaller. This suggests broad value in both graphics and vision applications on mobile and edge devices.

READ FULL TEXT
research
10/10/2020

Learning a model of facial shape and expression from 4D scans

The field of 3D face modeling has a large gap between high-end and low-e...
research
05/04/2023

Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization

There is a growing demand for the accessible creation of high-quality 3D...
research
09/14/2022

SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator

Recent years have seen growing interest in 3D human faces modelling due ...
research
11/11/2020

Survey on 3D face reconstruction from uncalibrated images

Recently, a lot of attention has been focused on the incorporation of 3D...
research
04/19/2023

ASM: Adaptive Skinning Model for High-Quality 3D Face Modeling

The research fields of parametric face models and 3D face reconstruction...
research
03/19/2018

Alive Caricature from 2D to 3D

Caricature is an art form that expresses subjects in abstract, simple an...
research
06/07/2017

DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

Face modeling has been paid much attention in the field of visual comput...

Please sign up or login with your details

Forgot password? Click here to reset