Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails

08/02/2021
by   Yang Zhang, et al.
2

In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.

READ FULL TEXT

page 1

page 7

page 10

page 11

page 14

page 20

page 23

page 24

research
02/09/2020

Face Hallucination with Finishing Touches

Obtaining a high-quality frontal face image from a low-resolution (LR) n...
research
11/29/2017

FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors

Face Super-Resolution (SR) is a domain-specific super-resolution problem...
research
11/19/2019

Joint Super-Resolution and Alignment of Tiny Faces

Super-resolution (SR) and landmark localization of tiny faces are highly...
research
02/29/2020

Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning

Previous research on face restoration often focused on repairing a speci...
research
04/16/2021

Accurate 3D Facial Geometry Prediction by Multi-Task, Multi-Modal, and Multi-Representation Landmark Refinement Network

This work focuses on complete 3D facial geometry prediction, including 3...
research
03/29/2020

Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation

Recent works based on deep learning and facial priors have succeeded in ...
research
06/11/2021

Attention-based Partial Face Recognition

Photos of faces captured in unconstrained environments, such as large cr...

Please sign up or login with your details

Forgot password? Click here to reset