Portrait Shadow Manipulation

by   Xuaner Cecilia Zhang, et al.

Casually-taken portrait photographs often suffer from unflattering lighting and shadowing because of suboptimal conditions in the environment. Aesthetic qualities such as the position and softness of shadows and the lighting ratio between the bright and dark parts of the face are frequently determined by the constraints of the environment rather than by the photographer. Professionals address this issue by adding light shaping tools such as scrims, bounce cards, and flashes. In this paper, we present a computational approach that gives casual photographers some of this control, thereby allowing poorly-lit portraits to be relit post-capture in a realistic and easily-controllable way. Our approach relies on a pair of neural networks—one to remove foreign shadows cast by external objects, and another to soften facial shadows cast by the features of the subject and to add a synthetic fill light to improve the lighting ratio. To train our first network we construct a dataset of real-world portraits wherein synthetic foreign shadows are rendered onto the face, and we show that our network learns to remove those unwanted shadows. To train our second network we use a dataset of Light Stage scans of human subjects to construct input/output pairs of input images harshly lit by a small light source, and variably softened and fill-lit output images of each face. We propose a way to explicitly encode facial symmetry and show that our dataset and training procedure enable the model to generalize to images taken in the wild. Together, these networks enable the realistic and aesthetically pleasing enhancement of shadows and lights in real-world portrait images



There are no comments yet.


page 1

page 5

page 6

page 8

page 9

page 10

page 11

page 12


A New Dimension in Testimony: Relighting Video with Reflectance Field Exemplars

We present a learning-based method for estimating 4D reflectance field o...

Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data

In portraits, eyeglasses may occlude facial regions and generate cast sh...

Deep Portrait Lighting Enhancement with 3D Guidance

Despite recent breakthroughs in deep learning methods for image lighting...

Towards High Fidelity Face Relighting with Realistic Shadows

Existing face relighting methods often struggle with two problems: maint...

Light Stage Super-Resolution: Continuous High-Frequency Relighting

The light stage has been widely used in computer graphics for the past t...

Single Image Portrait Relighting

Lighting plays a central role in conveying the essence and depth of the ...

DeepFlash: Turning a Flash Selfie into a Studio Portrait

We present a method for turning a flash selfie taken with a smartphone i...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.