DeepAI AI Chat
Log In Sign Up

Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering

03/28/2021
by   Fujun Luan, et al.
0

Reconstructing the shape and appearance of real-world objects using measured 2D images has been a long-standing problem in computer vision. In this paper, we introduce a new analysis-by-synthesis technique capable of producing high-quality reconstructions through robust coarse-to-fine optimization and physics-based differentiable rendering. Unlike most previous methods that handle geometry and reflectance largely separately, our method unifies the optimization of both by leveraging image gradients with respect to both object reflectance and geometry. To obtain physically accurate gradient estimates, we develop a new GPU-based Monte Carlo differentiable renderer leveraging recent advances in differentiable rendering theory to offer unbiased gradients while enjoying better performance than existing tools like PyTorch3D and redner. To further improve robustness, we utilize several shape and material priors as well as a coarse-to-fine optimization strategy to reconstruct geometry. We demonstrate that our technique can produce reconstructions with higher quality than previous methods such as COLMAP and Kinect Fusion.

READ FULL TEXT

page 2

page 4

page 5

page 7

page 8

06/07/2022

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

Recent advances in differentiable rendering have enabled high-quality re...
06/15/2022

Towards Differentiable Rendering for Sidescan Sonar Imagery

Recent advances in differentiable rendering, which allow calculating the...
06/10/2022

Differentiable Rendering of Neural SDFs through Reparameterization

We present a method to automatically compute correct gradients with resp...
05/30/2022

VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis

Differentiable rendering allows the application of computer graphics on ...
10/05/2021

Metameric Varifocal Holography

Computer-Generated Holography (CGH) offers the potential for genuine, hi...
04/26/2023

TextDeformer: Geometry Manipulation using Text Guidance

We present a technique for automatically producing a deformation of an i...