X-Section: Cross-section Prediction for Enhanced RGBD Fusion

03/03/2019
by   Andrea Nicastro, et al.
15

Detailed 3D reconstruction is an important challenge with application to robotics, augmented and virtual reality, which has seen impressive progress throughout the past years. Advancements were driven by the availability of depth cameras (RGB-D), as well as increased compute power, e.g. in the form of GPUs -- but also thanks to inclusion of machine learning in the process. Here, we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep learning to make object-level predictions about thicknesses that can be readily integrated into a volumetric multi-view fusion process, where we propose an extension to the popular KinectFusion approach. In essence, our method allows to complete shape in general indoor scenes behind what is sensed by the RGB-D camera, which may be crucial e.g. for robotic manipulation tasks or efficient scene exploration. Predicting object thicknesses rather than volumes allows us to work with comparably high spatial resolution without exploding memory and training data requirements on the employed Convolutional Neural Networks. In a series of qualitative and quantitative evaluations, we demonstrate how we accurately predict object thickness and reconstruct general 3D scenes containing multiple objects.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 8

page 9

page 10

research
07/18/2019

Temporally Coherent General Dynamic Scene Reconstruction

Existing techniques for dynamic scene reconstruction from multiple wide-...
research
04/09/2020

Neural Object Descriptors for Multi-View Shape Reconstruction

The choice of scene representation is crucial in both the shape inferenc...
research
09/14/2023

HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image

This paper presents a method to learn hand-object interaction prior for ...
research
12/09/2020

E3D: Event-Based 3D Shape Reconstruction

3D shape reconstruction is a primary component of augmented/virtual real...
research
05/23/2019

Bi-objective Framework for Sensor Fusion in RGB-D Multi-View Systems: Applications in Calibration

Complete and textured 3D reconstruction of dynamic scenes has been facil...
research
08/11/2020

Attention-based 3D Object Reconstruction from a Single Image

Recently, learning-based approaches for 3D reconstruction from 2D images...
research
10/13/2015

SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenes

We present an open-source, real-time implementation of SemanticPaint, a ...

Please sign up or login with your details

Forgot password? Click here to reset