Shape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering

by   Rui Li, et al.
King Abdullah University of Science and Technology

Simultaneous reconstruction of geometry and reflectance properties in uncontrolled environments remains a challenging problem. In this paper, we propose an efficient method to reconstruct the scene's 3D geometry and reflectance from multi-view photography using conventional hand-held cameras. Our method automatically builds a virtual scene in a differentiable rendering system that roughly matches the real world's scene parameters, optimized by minimizing photometric objectives alternatingly and stochastically. With the optimal scene parameters evaluated, photo-realistic novel views for various viewing angles and distances can then be generated by our approach. We present the results of captured scenes with complex geometry and various reflection types. Our method also shows superior performance compared to state-of-the-art alternatives in novel view synthesis visually and quantitatively.


page 5

page 6

page 7

page 8


Learning to Render Novel Views from Wide-Baseline Stereo Pairs

We introduce a method for novel view synthesis given only a single wide-...

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

We propose Differentiable Stereopsis, a multi-view stereo approach that ...

Towards Learning Neural Representations from Shadows

We present a method that learns neural scene representations from only s...

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

Expanding an existing tourist photo from a partially captured scene to a...

Radiative Transport Based Flame Volume Reconstruction from Videos

We introduce a novel approach for flame volume reconstruction from video...

NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields

Visually exploring in a real-world 4D spatiotemporal space freely in VR ...

Face Reflectance and Geometry Modeling via Differentiable Ray Tracing

We present a novel strategy to automatically reconstruct 3D faces from m...

Please sign up or login with your details

Forgot password? Click here to reset