Lighting, Reflectance and Geometry Estimation from 360^∘ Panoramic Stereo

04/20/2021
by   Junxuan Li, et al.
0

We propose a method for estimating high-definition spatially-varying lighting, reflectance, and geometry of a scene from 360^∘ stereo images. Our model takes advantage of the 360^∘ input to observe the entire scene with geometric detail, then jointly estimates the scene's properties with physical constraints. We first reconstruct a near-field environment light for predicting the lighting at any 3D location within the scene. Then we present a deep learning model that leverages the stereo information to infer the reflectance and surface normal. Lastly, we incorporate the physical constraints between lighting and geometry to refine the reflectance of the scene. Both quantitative and qualitative experiments show that our method, benefiting from the 360^∘ observation of the scene, outperforms prior state-of-the-art methods and enables more augmented reality applications such as mirror-objects insertion.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 6

page 7

page 8

03/18/2020

Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination

We present a deep learning solution for estimating the incident illumina...
12/11/2020

A Dark Flash Normal Camera

Casual photography is often performed in uncontrolled lighting that can ...
03/28/2018

Deep Photometric Stereo on a Sunny Day

Photometric Stereo in outdoor illumination remains a challenging, ill-po...
08/04/2017

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting

We introduce a novel method to obtain high-quality 3D reconstructions fr...
05/10/2019

SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks

This paper solves the Sparse Photometric stereo through Lighting Interpo...
03/30/2020

PointAR: Efficient Lighting Estimation for Mobile Augmented Reality

We propose an efficient lighting estimation pipeline that is suitable to...
11/29/2017

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

We present a novel 3D face reconstruction technique that leverages spars...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.