Deep Lighting Environment Map Estimation from Spherical Panoramas

05/16/2020
by   Vasileios Gkitsas, et al.
45

Estimating a scene's lighting is a very important task when compositing synthetic content within real environments, with applications in mixed reality and post-production. In this work we present a data-driven model that estimates an HDR lighting environment map from a single LDR monocular spherical panorama. In addition to being a challenging and ill-posed problem, the lighting estimation task also suffers from a lack of facile illumination ground truth data, a fact that hinders the applicability of data-driven methods. We approach this problem differently, exploiting the availability of surface geometry to employ image-based relighting as a data generator and supervision mechanism. This relies on a global Lambertian assumption that helps us overcome issues related to pre-baked lighting. We relight our training data and complement the model's supervision with a photometric loss, enabled by a differentiable image-based relighting technique. Finally, since we predict spherical spectral coefficients, we show that by imposing a distribution prior on the predicted coefficients, we can greatly boost performance. Code and models available at https://vcl3d.github.io/DeepPanoramaLighting.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 8

research
08/17/2021

Spatially and color consistent environment lighting estimation using deep neural networks for mixed reality

The representation of consistent mixed reality (XR) environments require...
research
09/16/2019

360^o Surface Regression with a Hyper-Sphere Loss

Omnidirectional vision is becoming increasingly relevant as more efficie...
research
03/18/2020

Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination

We present a deep learning solution for estimating the incident illumina...
research
02/12/2021

Outdoor inverse rendering from a single image using multiview self-supervision

In this paper we show how to perform scene-level inverse rendering to re...
research
12/06/2022

RANA: Relightable Articulated Neural Avatars

We propose RANA, a relightable and articulated neural avatar for the pho...
research
03/05/2018

Learning-Based Dequantization For Image Rsstoration Against Extremely Poor Illumination

All existing image enhancement methods, such as HDR tone mapping, cannot...
research
05/14/2017

A Closed-Form Model for Image-Based Distant Lighting

In this paper, we present a new mathematical foundation for image-based ...

Please sign up or login with your details

Forgot password? Click here to reset