Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization

05/04/2023
by   Connor Z. Lin, et al.
3

There is a growing demand for the accessible creation of high-quality 3D avatars that are animatable and customizable. Although 3D morphable models provide intuitive control for editing and animation, and robustness for single-view face reconstruction, they cannot easily capture geometric and appearance details. Methods based on neural implicit representations, such as signed distance functions (SDF) or neural radiance fields, approach photo-realism, but are difficult to animate and do not generalize well to unseen data. To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing. Trained from a collection of high-quality 3D scans, our face model is parameterized by geometry, expression, and texture latent codes with a learned SDF and explicit UV texture parameterization. Once trained, we can reconstruct an avatar from a single in-the-wild image by leveraging the learned prior to project the image into the latent space of our model. Our implicit morphable face models can be used to render an avatar from novel views, animate facial expressions by modifying expression codes, and edit textures by directly painting on the learned UV-texture maps. We demonstrate quantitatively and qualitatively that our method improves upon photo-realism, geometry, and expression accuracy compared to state-of-the-art methods.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 12

page 13

research
12/14/2021

I M Avatar: Implicit Morphable Head Avatars from Videos

Traditional morphable face models provide fine-grained control over expr...
research
07/01/2022

Neural Parameterization for Dynamic Human Head Editing

Implicit radiance functions emerged as a powerful scene representation f...
research
12/02/2022

ClipFace: Text-guided Editing of Textured 3D Morphable Models

We propose ClipFace, a novel self-supervised approach for text-guided ed...
research
04/22/2022

JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstruction

This paper addresses the problem of single view 3D human reconstruction....
research
04/28/2023

Learning Locally Editable Virtual Humans

In this paper, we propose a novel hybrid representation and end-to-end t...
research
07/14/2020

JNR: Joint-based Neural Rig Representation for Compact 3D Face Modeling

In this paper, we introduce a novel approach to learn a 3D face model us...
research
03/24/2023

TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images

Recent work in Neural Fields (NFs) learn 3D representations from class-s...

Please sign up or login with your details

Forgot password? Click here to reset