Sparse Needlets for Lighting Estimation with Spherical Transport Loss

06/24/2021
by   Fangneng Zhan, et al.
1

Accurate lighting estimation is challenging yet critical to many computer vision and computer graphics tasks such as high-dynamic-range (HDR) relighting. Existing approaches model lighting in either frequency domain or spatial domain which is insufficient to represent the complex lighting conditions in scenes and tends to produce inaccurate estimation. This paper presents NeedleLight, a new lighting estimation model that represents illumination with needlets and allows lighting estimation in both frequency domain and spatial domain jointly. An optimal thresholding function is designed to achieve sparse needlets which trims redundant lighting parameters and demonstrates superior localization properties for illumination representation. In addition, a novel spherical transport loss is designed based on optimal transport theory which guides to regress lighting representation parameters with consideration of the spatial information. Furthermore, we propose a new metric that is concise yet effective by directly evaluating the estimated illumination maps rather than rendered images. Extensive experiments show that NeedleLight achieves superior lighting estimation consistently across multiple evaluation metrics as compared with state-of-the-art methods.

READ FULL TEXT

page 2

page 5

page 6

page 7

02/20/2021

GMLight: Lighting Estimation via Geometric Distribution Approximation

Lighting estimation from a single image is an essential yet challenging ...
03/22/2022

Deep Portrait Delighting

We present a deep neural network for removing undesirable shading featur...
12/21/2020

EMLight: Lighting Estimation via Spherical Distribution Approximation

Illumination estimation from a single image is critical in 3D rendering ...
04/08/2019

Variational Uncalibrated Photometric Stereo under General Lighting

Photometric stereo (PS) techniques nowadays remain constrained to an ide...
07/29/2022

StyleLight: HDR Panorama Generation for Lighting Estimation and Editing

We present a new lighting estimation and editing framework to generate h...
03/30/2020

PointAR: Efficient Lighting Estimation for Mobile Augmented Reality

We propose an efficient lighting estimation pipeline that is suitable to...
07/19/2019

LightGuider: Guiding Interactive Lighting Design using Suggestions, Provenance, and Quality Visualization

LightGuider is a novel guidance-based approach to interactive lighting d...