Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images

08/07/2019
by   Yoshihiro Kanamori, et al.
2

Relighting of human images has various applications in image synthesis. For relighting, we must infer albedo, shape, and illumination from a human portrait. Previous techniques rely on human faces for this inference, based on spherical harmonics (SH) lighting. However, because they often ignore light occlusion, inferred shapes are biased and relit images are unnaturally bright particularly at hollowed regions such as armpits, crotches, or garment wrinkles. This paper introduces the first attempt to infer light occlusion in the SH formulation directly. Based on supervised learning using convolutional neural networks (CNNs), we infer not only an albedo map, illumination but also a light transport map that encodes occlusion as nine SH coefficients per pixel. The main difficulty in this inference is the lack of training datasets compared to unlimited variations of human portraits. Surprisingly, geometric information including occlusion can be inferred plausibly even with a small dataset of synthesized human figures, by carefully preparing the dataset so that the CNNs can exploit the data coherency. Our method accomplishes more realistic relighting than the occlusion-ignored formulation.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 9

page 10

research
07/23/2020

Pixel-Pair Occlusion Relationship Map(P2ORM): Formulation, Inference Application

We formalize concepts around geometric occlusion in 2D images (i.e., ign...
research
09/07/2020

Light Field View Synthesis via Aperture Flow and Propagation Confidence Map

This paper presents a learning-based approach to synthesize the view fro...
research
10/01/2020

X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation

We suggest to represent an X-Field -a set of 2D images taken across diff...
research
02/12/2021

Outdoor inverse rendering from a single image using multiview self-supervision

In this paper we show how to perform scene-level inverse rendering to re...
research
09/21/2023

ORTexME: Occlusion-Robust Human Shape and Pose via Temporal Average Texture and Mesh Encoding

In 3D human shape and pose estimation from a monocular video, models tra...
research
03/19/2016

Deep Shading: Convolutional Neural Networks for Screen-Space Shading

In computer vision, convolutional neural networks (CNNs) have recently a...
research
09/29/2017

A Variational Approach to Shape-from-shading Under Natural Illumination

A numerical solution to shape-from-shading under natural illumination is...

Please sign up or login with your details

Forgot password? Click here to reset