Neural Point Cloud Rendering via Multi-Plane Projection

12/10/2019
by   Xin Yu, et al.
28

We present a new deep point cloud rendering pipeline through multi-plane projections. The input to the network is the raw point cloud of a scene and the output are image or image sequences from a novel view or along a novel camera trajectory. Unlike previous approaches that directly project features from 3D points onto 2D image domain, we propose to project these features into a layered volume of camera frustum. In this way, the visibility of 3D points can be automatically learnt by the network, such that ghosting effects due to false visibility check as well as occlusions caused by noise interferences are both avoided successfully. Next, the 3D feature volume is fed into a 3D CNN to produce multiple layers of images w.r.t. the space division in the depth directions. The layered images are then blended based on learned weights to produce the final rendering results. Experiments show that our network produces more stable renderings compared to previous methods, especially near the object boundaries. Moreover, our pipeline is robust to noisy and relatively sparse point cloud for a variety of challenging scenes.

READ FULL TEXT

page 6

page 7

page 8

page 13

page 14

page 15

page 16

page 17

research
10/13/2021

ADOP: Approximate Differentiable One-Pixel Point Rendering

We present a novel point-based, differentiable neural rendering pipeline...
research
01/21/2022

Point-NeRF: Point-based Neural Radiance Fields

Volumetric neural rendering methods like NeRF generate high-quality view...
research
06/15/2022

Volume Conductor: Interactive Visibility Management for Crowded Volumes

We present a novel smart visibility system for visualizing crowded volum...
research
03/10/2023

3D Cinemagraphy from a Single Image

We present 3D Cinemagraphy, a new technique that marries 2D image animat...
research
10/07/2022

Multi-Frequency-Aware Patch Adversarial Learning for Neural Point Cloud Rendering

We present a neural point cloud rendering pipeline through a novel multi...
research
03/24/2022

NPBG++: Accelerating Neural Point-Based Graphics

We present a new system (NPBG++) for the novel view synthesis (NVS) task...
research
01/06/2003

Practical and Robust Stenciled Shadow Volumes for Hardware-Accelerated Rendering

Twenty-five years ago, Crow published the shadow volume approach for det...

Please sign up or login with your details

Forgot password? Click here to reset