Learning Spatial Attention for Face Super-Resolution

12/02/2020
by   Chaofeng Chen, et al.
58

General image super-resolution techniques have difficulties in recovering detailed face structures when applying to low resolution face images. Recent deep learning based methods tailored for face images have achieved improved performance by jointly trained with additional task such as face parsing and landmark prediction. However, multi-task learning requires extra manually labeled data. Besides, most of the existing works can only generate relatively low resolution face images (e.g., 128×128), and their applications are therefore limited. In this paper, we introduce a novel SPatial Attention Residual Network (SPARNet) built on our newly proposed Face Attention Units (FAUs) for face super-resolution. Specifically, we introduce a spatial attention mechanism to the vanilla residual blocks. This enables the convolutional layers to adaptively bootstrap features related to the key face structures and pay less attention to those less feature-rich regions. This makes the training more effective and efficient as the key face structures only account for a very small portion of the face image. Visualization of the attention maps shows that our spatial attention network can capture the key face structures well even for very low resolution faces (e.g., 16×16). Quantitative comparisons on various kinds of metrics (including PSNR, SSIM, identity similarity, and landmark detection) demonstrate the superiority of our method over current state-of-the-arts. We further extend SPARNet with multi-scale discriminators, named as SPARNetHD, to produce high resolution results (i.e., 512×512). We show that SPARNetHD trained with synthetic data cannot only produce high quality and high resolution outputs for synthetically degraded face images, but also show good generalization ability to real world low quality face images.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 10

page 11

page 13

research
01/11/2021

Deep Learning-based Face Super-resolution: A Survey

Face super-resolution, also known as face hallucination, which is aimed ...
research
11/04/2019

FCSR-GAN: Joint Face Completion and Super-resolution via Multi-task Learning

Combined variations containing low-resolution and occlusion often presen...
research
04/06/2023

Super-Resolving Face Image by Facial Parsing Information

Face super-resolution is a technology that transforms a low-resolution f...
research
03/25/2021

JDSR-GAN: Constructing A Joint and Collaborative Learning Network for Masked Face Super-Resolution

With the growing importance of preventing the COVID-19 virus, face image...
research
12/21/2018

Face Hallucination Revisited: An Exploratory Study on Dataset Bias

Contemporary face hallucination (FH) models exhibit considerable ability...
research
08/20/2019

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Recent studies have shown remarkable success in face manipulation task w...
research
10/19/2019

Component Attention Guided Face Super-Resolution Network: CAGFace

To make the best use of the underlying structure of faces, the collectiv...

Please sign up or login with your details

Forgot password? Click here to reset