Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video

10/12/2022
by   Xuan Gao, et al.
0

We present a novel semantic model for human head defined with neural radiance field. The 3D-consistent head model consist of a set of disentangled and interpretable bases, and can be driven by low-dimensional expression coefficients. Thanks to the powerful representation ability of neural radiance field, the constructed model can represent complex facial attributes including hair, wearings, which can not be represented by traditional mesh blendshape. To construct the personalized semantic facial model, we propose to define the bases as several multi-level voxel fields. With a short monocular RGB video as input, our method can construct the subject's semantic facial NeRF model with only ten to twenty minutes, and can render a photo-realistic human head image in tens of miliseconds with a given expression coefficient and view direction. With this novel representation, we apply it to many tasks like facial retargeting and expression editing. Experimental results demonstrate its strong representation ability and training/inference speed. Demo videos and released code are provided in our project page: https://ustc3dv.github.io/NeRFBlendShape/

READ FULL TEXT

page 7

page 8

page 9

page 10

page 11

research
06/13/2022

RigNeRF: Fully Controllable Neural 3D Portraits

Volumetric neural rendering methods, such as neural radiance fields (NeR...
research
11/23/2022

ManVatar : Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels

With NeRF widely used for facial reenactment, recent methods can recover...
research
11/28/2022

High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priors

High-fidelity facial avatar reconstruction from a monocular video is a s...
research
05/17/2023

LPMM: Intuitive Pose Control for Neural Talking-Head Model via Landmark-Parameter Morphable Model

While current talking head models are capable of generating photorealist...
research
04/21/2023

Implicit Neural Head Synthesis via Controllable Local Deformation Fields

High-quality reconstruction of controllable 3D head avatars from 2D vide...
research
11/15/2021

Deep Semantic Manipulation of Facial Videos

Editing and manipulating facial features in videos is an interesting and...
research
06/01/2023

AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars

Capturing and editing full head performances enables the creation of vir...

Please sign up or login with your details

Forgot password? Click here to reset