Face De-occlusion using 3D Morphable Model and Generative Adversarial Network

04/12/2019
by   Xiaowei Yuan, et al.
0

In recent decades, 3D morphable model (3DMM) has been commonly used in image-based photorealistic 3D face reconstruction. However, face images are often corrupted by serious occlusion by non-face objects including eyeglasses, masks, and hands. Such objects block the correct capture of landmarks and shading information. Therefore, the reconstructed 3D face model is hardly reusable. In this paper, a novel method is proposed to restore de-occluded face images based on inverse use of 3DMM and generative adversarial network. We utilize the 3DMM prior to the proposed adversarial network and combine a global and local adversarial convolutional neural network to learn face de-occlusion model. The 3DMM serves not only as geometric prior but also proposes the face region for the local discriminator. Experiment results confirm the effectiveness and robustness of the proposed algorithm in removing challenging types of occlusions with various head poses and illumination. Furthermore, the proposed method reconstructs the correct 3D face model with de-occluded textures.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

research
12/15/2021

Segmentation-Reconstruction-Guided Facial Image De-occlusion

Occlusions are very common in face images in the wild, leading to the de...
research
08/21/2021

End2End Occluded Face Recognition by Masking Corrupted Features

With the recent advancement of deep convolutional neural networks, signi...
research
07/13/2020

DETCID: Detection of Elongated Touching Cells with Inhomogeneous Illumination using a Deep Adversarial Network

Clostridioides difficile infection (C. diff) is the most common cause of...
research
11/09/2021

SAFA: Structure Aware Face Animation

Recent success of generative adversarial networks (GAN) has made great p...
research
10/20/2019

LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network

In this paper, we explore the task of generating photo-realistic face im...
research
02/19/2020

SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion

Certain facial parts are salient (unique) in appearance, which substanti...
research
11/30/2018

FSNet: An Identity-Aware Generative Model for Image-based Face Swapping

This paper presents FSNet, a deep generative model for image-based face ...

Please sign up or login with your details

Forgot password? Click here to reset