NeuTex: Neural Texture Mapping for Volumetric Neural Rendering

03/01/2021
by   Fanbo Xiang, et al.
0

Recent work has demonstrated that volumetric scene representations combined with differentiable volume rendering can enable photo-realistic rendering for challenging scenes that mesh reconstruction fails on. However, these methods entangle geometry and appearance in a "black-box" volume that cannot be edited. Instead, we present an approach that explicitly disentangles geometry–represented as a continuous 3D volume–from appearance–represented as a continuous 2D texture map. We achieve this by introducing a 3D-to-2D texture mapping (or surface parameterization) network into volumetric representations. We constrain this texture mapping network using an additional 2D-to-3D inverse mapping network and a novel cycle consistency loss to make 3D surface points map to 2D texture points that map back to the original 3D points. We demonstrate that this representation can be reconstructed using only multi-view image supervision and generates high-quality rendering results. More importantly, by separating geometry and texture, we allow users to edit appearance by simply editing 2D texture maps.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 11

page 12

page 13

research
07/01/2022

Neural Parameterization for Dynamic Human Head Editing

Implicit radiance functions emerged as a powerful scene representation f...
research
03/29/2022

DRaCoN – Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars

Acquisition and creation of digital human avatars is an important proble...
research
03/29/2022

Towards Learning Neural Representations from Shadows

We present a method that learns neural scene representations from only s...
research
12/19/2022

Real-Time Rendering of Arbitrary Surface Geometries using Learnt Transfer

Precomputed Radiance Transfer (PRT) is widely used for real-time photore...
research
11/30/2020

Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping

We introduce VERTEX, an effective solution to recover 3D shape and intri...
research
08/15/2022

Adaptive Joint Optimization for 3D Reconstruction with Differentiable Rendering

Due to inevitable noises introduced during scanning and quantization, 3D...
research
06/10/2022

Differentiable Rendering of Neural SDFs through Reparameterization

We present a method to automatically compute correct gradients with resp...

Please sign up or login with your details

Forgot password? Click here to reset