Towards High Fidelity Face Relighting with Realistic Shadows

04/02/2021
by   Andrew Hou, et al.
0

Existing face relighting methods often struggle with two problems: maintaining the local facial details of the subject and accurately removing and synthesizing shadows in the relit image, especially hard shadows. We propose a novel deep face relighting method that addresses both problems. Our method learns to predict the ratio (quotient) image between a source image and the target image with the desired lighting, allowing us to relight the image while maintaining the local facial details. During training, our model also learns to accurately modify shadows by using estimated shadow masks to emphasize on the high-contrast shadow borders. Furthermore, we introduce a method to use the shadow mask to estimate the ambient light intensity in an image, and are thus able to leverage multiple datasets during training with different global lighting intensities. With quantitative and qualitative evaluations on the Multi-PIE and FFHQ datasets, we demonstrate that our proposed method faithfully maintains the local facial details of the subject and can accurately handle hard shadows while achieving state-of-the-art face relighting performance.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

page 11

page 12

page 13

03/30/2022

Face Relighting with Geometrically Consistent Shadows

Most face relighting methods are able to handle diffuse shadows, but str...
09/19/2022

BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction

We propose BareSkinNet, a novel method that simultaneously removes makeu...
09/16/2021

TANet: A new Paradigm for Global Face Super-resolution via Transformer-CNN Aggregation Network

Recently, face super-resolution (FSR) methods either feed whole face ima...
01/10/2019

Learning Continuous Face Age Progression: A Pyramid of GANs

The two underlying requirements of face age progression, i.e. aging accu...
12/14/2020

FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction

Facial Expressions induce a variety of high-level details on the 3D face...
05/18/2020

Portrait Shadow Manipulation

Casually-taken portrait photographs often suffer from unflattering light...
03/22/2022

Deep Portrait Delighting

We present a deep neural network for removing undesirable shading featur...