Neural Point-Based Graphics

06/19/2019
by   Kara-Ali Aliev, et al.
1

We present a new point-based approach for modeling complex scenes. The approach uses a raw point cloud as the geometric representation of a scene, and augments each point with a learnable neural descriptor that encodes local geometry and appearance. A deep rendering network is learned in parallel with the descriptors, so that new views of the scene can be obtained by passing the rasterizations of a point cloud from new viewpoints through this network. The input rasterizations use the learned descriptors as point pseudo-colors. We show that the proposed approach can be used for modeling complex scenes and obtaining their photorealistic views, while avoiding explicit surface estimation and meshing. In particular, compelling results are obtained for scene scanned using hand-held commodity RGB-D sensors as well as standard RGB cameras even in the presence of objects that are challenging for standard mesh-based modeling.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

page 8

page 10

research
09/06/2020

TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer

We propose and evaluate a neural point-based graphics method that can mo...
research
09/24/2017

3D Camouflaging Object using RGB-D Sensors

This paper proposes a new optical camouflage system that uses RGB-D came...
research
07/21/2022

MeshLoc: Mesh-Based Visual Localization

Visual localization, i.e., the problem of camera pose estimation, is a c...
research
03/24/2022

NPBG++: Accelerating Neural Point-Based Graphics

We present a new system (NPBG++) for the novel view synthesis (NVS) task...
research
02/23/2021

UnsupervisedR R: Unsupervised Point Cloud Registration via Differentiable Rendering

Aligning partial views of a scene into a single whole is essential to un...
research
11/22/2011

Contextually Guided Semantic Labeling and Search for 3D Point Clouds

RGB-D cameras, which give an RGB image to- gether with depths, are becom...
research
04/16/2021

Point-Based Modeling of Human Clothing

We propose a new approach to human clothing modeling based on point clou...

Please sign up or login with your details

Forgot password? Click here to reset