Portrait Shadow Manipulation

by   Xuaner Cecilia Zhang, et al.

Casually-taken portrait photographs often suffer from unflattering lighting and shadowing because of suboptimal conditions in the environment. Aesthetic qualities such as the position and softness of shadows and the lighting ratio between the bright and dark parts of the face are frequently determined by the constraints of the environment rather than by the photographer. Professionals address this issue by adding light shaping tools such as scrims, bounce cards, and flashes. In this paper, we present a computational approach that gives casual photographers some of this control, thereby allowing poorly-lit portraits to be relit post-capture in a realistic and easily-controllable way. Our approach relies on a pair of neural networks—one to remove foreign shadows cast by external objects, and another to soften facial shadows cast by the features of the subject and to add a synthetic fill light to improve the lighting ratio. To train our first network we construct a dataset of real-world portraits wherein synthetic foreign shadows are rendered onto the face, and we show that our network learns to remove those unwanted shadows. To train our second network we use a dataset of Light Stage scans of human subjects to construct input/output pairs of input images harshly lit by a small light source, and variably softened and fill-lit output images of each face. We propose a way to explicitly encode facial symmetry and show that our dataset and training procedure enable the model to generalize to images taken in the wild. Together, these networks enable the realistic and aesthetically pleasing enhancement of shadows and lights in real-world portrait images


page 1

page 5

page 6

page 8

page 9

page 10

page 11

page 12


Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation

Given a portrait image of a person and an environment map of the target ...

A New Dimension in Testimony: Relighting Video with Reflectance Field Exemplars

We present a learning-based method for estimating 4D reflectance field o...

Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data

In portraits, eyeglasses may occlude facial regions and generate cast sh...

DiFaReli: Diffusion Face Relighting

We present a novel approach to single-view face relighting in the wild. ...

Deep Portrait Lighting Enhancement with 3D Guidance

Despite recent breakthroughs in deep learning methods for image lighting...

Towards High Fidelity Face Relighting with Realistic Shadows

Existing face relighting methods often struggle with two problems: maint...

Light Stage Super-Resolution: Continuous High-Frequency Relighting

The light stage has been widely used in computer graphics for the past t...

Please sign up or login with your details

Forgot password? Click here to reset