Face Relighting with Geometrically Consistent Shadows

03/30/2022
by   Andrew Hou, et al.
0

Most face relighting methods are able to handle diffuse shadows, but struggle to handle hard shadows, such as those cast by the nose. Methods that propose techniques for handling hard shadows often do not produce geometrically consistent shadows since they do not directly leverage the estimated face geometry while synthesizing them. We propose a novel differentiable algorithm for synthesizing hard shadows based on ray tracing, which we incorporate into training our face relighting model. Our proposed algorithm directly utilizes the estimated face geometry to synthesize geometrically consistent hard shadows. We demonstrate through quantitative and qualitative experiments on Multi-PIE and FFHQ that our method produces more geometrically consistent shadows than previous face relighting methods while also achieving state-of-the-art face relighting performance under directional lighting. In addition, we demonstrate that our differentiable hard shadow modeling improves the quality of the estimated face geometry over diffuse shading models.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 11

page 12

page 13

research
04/02/2021

Towards High Fidelity Face Relighting with Realistic Shadows

Existing face relighting methods often struggle with two problems: maint...
research
01/13/2021

Practical Face Reconstruction via Differentiable Ray Tracing

We present a differentiable ray-tracing based novel face reconstruction ...
research
03/29/2021

Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

Robust face reconstruction from monocular image in general lighting cond...
research
07/11/2022

Geometry-aware Single-image Full-body Human Relighting

Single-image human relighting aims to relight a target human under new l...
research
08/08/2018

Adversarial Geometry and Lighting using a Differentiable Renderer

Many machine learning classifiers are vulnerable to adversarial attacks,...
research
10/03/2019

Face Reflectance and Geometry Modeling via Differentiable Ray Tracing

We present a novel strategy to automatically reconstruct 3D faces from m...
research
06/22/2018

KinshipGAN: Synthesizing of Kinship Faces From Family Photos by Regularizing a Deep Face Network

In this paper, we propose a kinship generator network that can synthesiz...

Please sign up or login with your details

Forgot password? Click here to reset