DeepAI AI Chat
Log In Sign Up

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

12/08/2021
by   Mingfei Chen, et al.
0

In this work we develop a generalizable and efficient Neural Radiance Field (NeRF) pipeline for high-fidelity free-viewpoint human body synthesis under settings with sparse camera views. Though existing NeRF-based methods can synthesize rather realistic details for human body, they tend to produce poor results when the input has self-occlusion, especially for unseen humans under sparse views. Moreover, these methods often require a large number of sampling points for rendering, which leads to low efficiency and limits their real-world applicability. To address these challenges, we propose a Geometry-guided Progressive NeRF (GP-NeRF). In particular, to better tackle self-occlusion, we devise a geometry-guided multi-view feature integration approach that utilizes the estimated geometry prior to integrate the incomplete information from input views and construct a complete geometry volume for the target human body. Meanwhile, for achieving higher rendering efficiency, we introduce a progressive rendering pipeline through geometry guidance, which leverages the geometric feature volume and the predicted density values to progressively reduce the number of sampling points and speed up the rendering process. Experiments on the ZJU-MoCap and THUman datasets show that our method outperforms the state-of-the-arts significantly across multiple generalization settings, while the time cost is reduced > 70 progressive rendering pipeline.

READ FULL TEXT

page 3

page 8

page 11

01/20/2022

HDhuman: High-quality Human Performance Capture with Sparse Views

In this paper, we introduce HDhuman, a method that addresses the challen...
06/07/2021

DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Rendering

We introduce DoubleField, a novel representation combining the merits of...
03/24/2023

GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images

In this work, we focus on synthesizing high-fidelity novel view images f...
12/17/2020

Learning Compositional Radiance Fields of Dynamic Human Heads

Photorealistic rendering of dynamic humans is an important ability for t...
06/17/2021

Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering

Image-based geometric modeling and novel view synthesis based on sparse,...
12/15/2022

SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory

Neural Radiance Fields (NeRF) have demonstrated superior novel view synt...
07/14/2022

Relighting4D: Neural Relightable Human from Videos

Human relighting is a highly desirable yet challenging task. Existing wo...