TensoIR: Tensorial Inverse Rendering

by   Haian Jin, et al.

We propose TensoIR, a novel inverse rendering approach based on tensor factorization and neural fields. Unlike previous works that use purely MLP-based neural fields, thus suffering from low capacity and high computation costs, we extend TensoRF, a state-of-the-art approach for radiance field modeling, to estimate scene geometry, surface reflectance, and environment illumination from multi-view images captured under unknown lighting conditions. Our approach jointly achieves radiance field reconstruction and physically-based model estimation, leading to photo-realistic novel view synthesis and relighting results. Benefiting from the efficiency and extensibility of the TensoRF-based representation, our method can accurately model secondary shading effects (like shadows and indirect lighting) and generally support input images captured under single or multiple unknown lighting conditions. The low-rank tensor representation allows us to not only achieve fast and compact reconstruction but also better exploit shared information under an arbitrary number of capturing lighting conditions. We demonstrate the superiority of our method to baseline methods qualitatively and quantitatively on various challenging synthetic and real-world scenes.


page 1

page 3

page 5

page 7

page 8

page 11

page 13

page 14


Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes

We present a multi-view inverse rendering method for large-scale real-wo...

PANDORA: Polarization-Aided Neural Decomposition Of Radiance

Reconstructing an object's geometry and appearance from multiple images,...

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

We address the problem of recovering the shape and spatially-varying ref...

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis

The advancement of generative radiance fields has pushed the boundary of...

Signed Distance Fields Dynamic Diffuse Global Illumination

Global Illumination (GI) is of utmost importance in the field of photo-r...

WildLight: In-the-wild Inverse Rendering with a Flashlight

This paper proposes a practical photometric solution for the challenging...

NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis

We present a method that takes as input a set of images of a scene illum...

Please sign up or login with your details

Forgot password? Click here to reset