Convolutional Neural Opacity Radiance Fields

04/05/2021
by   Haimin Luo, et al.
0

Photo-realistic modeling and rendering of fuzzy objects with complex opacity are critical for numerous immersive VR/AR applications, but it suffers from strong view-dependent brightness, color. In this paper, we propose a novel scheme to generate opacity radiance fields with a convolutional neural renderer for fuzzy objects, which is the first to combine both explicit opacity supervision and convolutional mechanism into the neural radiance field framework so as to enable high-quality appearance and global consistent alpha mattes generation in arbitrary novel views. More specifically, we propose an efficient sampling strategy along with both the camera rays and image plane, which enables efficient radiance field sampling and learning in a patch-wise manner, as well as a novel volumetric feature integration scheme that generates per-patch hybrid feature embeddings to reconstruct the view-consistent fine-detailed appearance and opacity output. We further adopt a patch-wise adversarial training scheme to preserve both high-frequency appearance and opacity details in a self-supervised framework. We also introduce an effective multi-view image capture system to capture high-quality color and alpha maps for challenging fuzzy objects. Extensive experiments on existing and our new challenging fuzzy object dataset demonstrate that our method achieves photo-realistic, globally consistent, and fined detailed appearance and opacity free-viewpoint rendering for various fuzzy objects.

READ FULL TEXT

page 2

page 3

page 6

page 7

page 8

page 10

page 12

research
02/17/2022

MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging

Photo-realistic neural reconstruction and rendering of the human portrai...
research
04/06/2021

MirrorNeRF: One-shot Neural Portrait RadianceField from Multi-mirror Catadioptric Imaging

Photo-realistic neural reconstruction and rendering of the human portrai...
research
04/06/2023

Instant-NVR: Instant Neural Volumetric Rendering for Human-object Interactions from Monocular RGBD Stream

Convenient 4D modeling of human-object interactions is essential for num...
research
12/10/2022

HumanGen: Generating Human Radiance Fields with Explicit Priors

Recent years have witnessed the tremendous progress of 3D GANs for gener...
research
02/11/2022

Artemis: Articulated Neural Pets with Appearance and Motion synthesis

We, humans, are entering into a virtual era and indeed want to bring ani...
research
10/22/2022

NeARportation: A Remote Real-time Neural Rendering Framework

While the presentation of photo-realistic appearance plays a major role ...
research
07/14/2021

Few-shot Neural Human Performance Rendering from Sparse RGBD Videos

Recent neural rendering approaches for human activities achieve remarkab...

Please sign up or login with your details

Forgot password? Click here to reset