Shape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering

10/25/2021
by   Rui Li, et al.
King Abdullah University of Science and Technology
0

Simultaneous reconstruction of geometry and reflectance properties in uncontrolled environments remains a challenging problem. In this paper, we propose an efficient method to reconstruct the scene's 3D geometry and reflectance from multi-view photography using conventional hand-held cameras. Our method automatically builds a virtual scene in a differentiable rendering system that roughly matches the real world's scene parameters, optimized by minimizing photometric objectives alternatingly and stochastically. With the optimal scene parameters evaluated, photo-realistic novel views for various viewing angles and distances can then be generated by our approach. We present the results of captured scenes with complex geometry and various reflection types. Our method also shows superior performance compared to state-of-the-art alternatives in novel view synthesis visually and quantitatively.

READ FULL TEXT

page 5

page 6

page 7

page 8

04/17/2023

Learning to Render Novel Views from Wide-Baseline Stereo Pairs

We introduce a method for novel view synthesis given only a single wide-...
10/11/2021

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

We propose Differentiable Stereopsis, a multi-view stereo approach that ...
03/29/2022

Towards Learning Neural Representations from Shadows

We present a method that learns neural scene representations from only s...
07/14/2022

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

Expanding an existing tourist photo from a partially captured scene to a...
09/17/2018

Radiative Transport Based Flame Volume Reconstruction from Videos

We introduce a novel approach for flame volume reconstruction from video...
10/28/2022

NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields

Visually exploring in a real-world 4D spatiotemporal space freely in VR ...
10/03/2019

Face Reflectance and Geometry Modeling via Differentiable Ray Tracing

We present a novel strategy to automatically reconstruct 3D faces from m...

Please sign up or login with your details

Forgot password? Click here to reset