Face Reflectance and Geometry Modeling via Differentiable Ray Tracing

10/03/2019
by   Abdallah Dib, et al.
0

We present a novel strategy to automatically reconstruct 3D faces from monocular images with explicitly disentangled facial geometry (pose, identity and expression), reflectance (diffuse and specular albedo), and self-shadows. The scene lights are modeled as a virtual light stage with pre-oriented area lights used in conjunction with differentiable Monte-Carlo ray tracing to optimize the scene and face parameters. With correctly disentangled self-shadows and specular reflection parameters, we can not only obtain robust facial geometry reconstruction, but also gain explicit control over these parameters, with several practical applications. We can change facial expressions with accurate resultant self-shadows or relight the scene and obtain accurate specular reflection and several other parameter combinations.

READ FULL TEXT
research
01/13/2021

Practical Face Reconstruction via Differentiable Ray Tracing

We present a differentiable ray-tracing based novel face reconstruction ...
research
08/07/2023

Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing

Recently, Neural Radiance Fields (NeRF) has exhibited significant succes...
research
03/29/2021

Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

Robust face reconstruction from monocular image in general lighting cond...
research
08/24/2020

Monocular Reconstruction of Neural Face Reflectance Fields

The reflectance field of a face describes the reflectance properties res...
research
10/04/2020

Learning Complete 3D Morphable Face Models from Images and Videos

Most 3D face reconstruction methods rely on 3D morphable models, which d...
research
10/25/2021

Shape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering

Simultaneous reconstruction of geometry and reflectance properties in un...
research
03/30/2022

Face Relighting with Geometrically Consistent Shadows

Most face relighting methods are able to handle diffuse shadows, but str...

Please sign up or login with your details

Forgot password? Click here to reset