DeepAI AI Chat
Log In Sign Up

Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images

by   Harim Jung, et al.

The 3D Morphable Model (3DMM), which is a Principal Component Analysis (PCA) based statistical model that represents a 3D face using linear basis functions, has shown promising results for reconstructing 3D faces from single-view in-the-wild images. However, 3DMM has restricted representation power due to the limited number of 3D scans and the global linear basis. To address the limitations of 3DMM, we propose a straightforward learning-based method that reconstructs a 3D face mesh through Free-Form Deformation (FFD) for the first time. FFD is a geometric modeling method that embeds a reference mesh within a parallelepiped grid and deforms the mesh by moving the sparse control points of the grid. As FFD is based on mathematically defined basis functions, it has no limitation in representation power. Thus, we can recover accurate 3D face meshes by estimating appropriate deviation of control points as deformation parameters. Although both 3DMM and FFD are parametric models, it is difficult to predict the effect of the 3DMM parameters on the face shape, while the deformation parameters of FFD are interpretable in terms of their effect on the final shape of the mesh. This practical advantage of FFD allows the resulting mesh and control points to serve as a good starting point for 3D face modeling, in that ordinary users can fine-tune the mesh by using widely available 3D software tools. Experiments on multiple datasets demonstrate how our method successfully estimates the 3D face geometry and facial expressions from 2D face images, achieving comparable performance to the state-of-the-art methods.


page 1

page 5


On Learning 3D Face Morphable Model from In-the-wild Images

As a classic statistical model of 3D facial shape and albedo, 3D Morphab...

Alive Caricature from 2D to 3D

Caricature is an art form that expresses subjects in abstract, simple an...

Mesh-based Autoencoders for Localized Deformation Component Analysis

Spatially localized deformation components are very useful for shape ana...

Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

3D Morphable Models (3DMMs) are statistical models that represent facial...

3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction

Caricature is an artistic representation that deliberately exaggerates t...

Clustering of the Blendshape Facial Model

Digital human animation relies on high-quality 3D models of the human fa...

Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild

We propose an end-to-end deep-learning approach for automatic rigging an...