Learning Oracle Attention for High-fidelity Face Completion

by   Tong Zhou, et al.

High-fidelity face completion is a challenging task due to the rich and subtle facial textures involved. What makes it more complicated is the correlations between different facial components, for example, the symmetry in texture and structure between both eyes. While recent works adopted the attention mechanism to learn the contextual relations among elements of the face, they have largely overlooked the disastrous impacts of inaccurate attention scores; in addition, they fail to pay sufficient attention to key facial components, the completion results of which largely determine the authenticity of a face image. Accordingly, in this paper, we design a comprehensive framework for face completion based on the U-Net structure. Specifically, we propose a dual spatial attention module to efficiently learn the correlations between facial textures at multiple scales; moreover, we provide an oracle supervision signal to the attention module to ensure that the obtained attention scores are reasonable. Furthermore, we take the location of the facial components as prior knowledge and impose a multi-discriminator on these regions, with which the fidelity of facial components is significantly promoted. Extensive experiments on two high-resolution face datasets including CelebA-HQ and Flickr-Faces-HQ demonstrate that the proposed approach outperforms state-of-the-art methods by large margins.


page 12

page 13

page 14

page 15

page 16

page 17

page 18

page 19


Self-supervised High-fidelity and Re-renderable 3D Facial Reconstruction from a Single Image

Reconstructing high-fidelity 3D facial texture from a single image is a ...

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

In the past few years, a lot of work has been done towards reconstructin...

Unsupervised High-Fidelity Facial Texture Generation and Reconstruction

Many methods have been proposed over the years to tackle the task of fac...

Photorealistic Facial Texture Inference Using Deep Neural Networks

We present a data-driven inference method that can synthesize a photorea...

Fine-Grained Face Swapping via Regional GAN Inversion

We present a novel paradigm for high-fidelity face swapping that faithfu...

CPNet: Exploiting CLIP-based Attention Condenser and Probability Map Guidance for High-fidelity Talking Face Generation

Recently, talking face generation has drawn ever-increasing attention fr...

Kinship Verification from rPPG using 1DCNN Attention networks

Facial kinship verification aims at automatically determining whether tw...

Please sign up or login with your details

Forgot password? Click here to reset