Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

06/15/2021
by   Zhenyu Zhang, et al.
0

Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. While plausible facial details are predicted, the models tend to over-depend on local color appearance and suffer from ambiguous noise. To address such problem, this paper presents a novel Learning to Aggregate and Personalize (LAP) framework for unsupervised robust 3D face modeling. Instead of using controlled environment, the proposed method implicitly disentangles ID-consistent and scene-specific face from unconstrained photo set. Specifically, to learn ID-consistent face, LAP adaptively aggregates intrinsic face factors of an identity based on a novel curriculum learning approach with relaxed consistency loss. To adapt the face for a personalized scene, we propose a novel attribute-refining network to modify ID-consistent face with target attribute and details. Based on the proposed method, we make unsupervised 3D face modeling benefit from meaningful image facial structure and possibly higher resolutions. Extensive experiments on benchmarks show LAP recovers superior or competitive face shape and texture, compared with state-of-the-art (SOTA) methods with or without prior and supervision.

READ FULL TEXT

page 4

page 6

page 8

research
06/14/2017

Photo-realistic Facial Texture Transfer

Style transfer methods have achieved significant success in recent years...
research
12/27/2022

DiffFace: Diffusion-based Face Swapping with Facial Guidance

In this paper, we propose a diffusion-based face swapping framework for ...
research
06/18/2021

HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping

In this work, we propose a high fidelity face swapping method, called Hi...
research
07/24/2019

Pose-variant 3D Facial Attribute Generation

We address the challenging problem of generating facial attributes using...
research
04/09/2019

Towards High-fidelity Nonlinear 3D Face Morphable Model

Embedding 3D morphable basis functions into deep neural networks opens g...
research
04/07/2021

Everything's Talkin': Pareidolia Face Reenactment

We present a new application direction named Pareidolia Face Reenactment...
research
07/03/2016

Robust Deep Appearance Models

This paper presents a novel Robust Deep Appearance Models to learn the n...

Please sign up or login with your details

Forgot password? Click here to reset