DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Rendering

06/07/2021
by   Ruizhi Shao, et al.
7

We introduce DoubleField, a novel representation combining the merits of both surface field and radiance field for high-fidelity human rendering. Within DoubleField, the surface field and radiance field are associated together by a shared feature embedding and a surface-guided sampling strategy. In this way, DoubleField has a continuous but disentangled learning space for geometry and appearance modeling, which supports fast training, inference, and finetuning. To achieve high-fidelity free-viewpoint rendering, DoubleField is further augmented to leverage ultra-high-resolution inputs, where a view-to-view transformer and a transfer learning scheme are introduced for more efficient learning and finetuning from sparse-view inputs at original resolutions. The efficacy of DoubleField is validated by the quantitative evaluations on several datasets and the qualitative results in a real-world sparse multi-view system, showing its superior capability for photo-realistic free-viewpoint human rendering. For code and demo video, please refer to our project page: http://www.liuyebin.com/dbfield/dbfield.html.

READ FULL TEXT

page 1

page 8

page 9

page 10

research
12/06/2021

HumanNeRF: Generalizable Neural Human Radiance Field from Sparse Inputs

Recent neural human representations can produce high-quality multi-view ...
research
10/15/2018

Deep Surface Light Fields

A surface light field represents the radiance of rays originating from a...
research
12/08/2021

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

In this work we develop a generalizable and efficient Neural Radiance Fi...
research
12/09/2022

4K-NeRF: High Fidelity Neural Radiance Fields at Ultra High Resolutions

In this paper, we present a novel and effective framework, named 4K-NeRF...
research
06/17/2022

Efficient Raycasting of View-Dependent Piecewise Constant Representations of Volumetric Data

We present an efficient raycasting-based rendering algorithm for view-de...
research
10/21/2022

HDHumans: A Hybrid Approach for High-fidelity Digital Humans

Photo-real digital human avatars are of enormous importance in graphics,...
research
10/04/2022

SelfNeRF: Fast Training NeRF for Human from Monocular Self-rotating Video

In this paper, we propose SelfNeRF, an efficient neural radiance field b...

Please sign up or login with your details

Forgot password? Click here to reset