Fast Generation of High Fidelity RGB-D Images by Deep-Learning with Adaptive Convolution

02/12/2020
by   Chuhua Xian, et al.
0

Using the raw data from consumer-level RGB-D cameras as input, we propose a deep-learning based approach to efficiently generate RGB-D images with completed information in high resolution. To process the input images in low resolution with missing regions, new operators for adaptive convolution are introduced in our deep-learning network that consists of three cascaded modules – the completion module, the refinement module and the super-resolution module. The completion module is based on an architecture of encoder-decoder, where the features of input raw RGB-D will be automatically extracted by the encoding layers of a deep neural-network. The decoding layers are applied to reconstruct the completed depth map, which is followed by a refinement module to sharpen the boundary of different regions. For the super-resolution module, we generate RGB-D images in high resolution by multiple layers for feature extraction and a layer for up-sampling. Benefited from the adaptive convolution operators newly proposed in this paper, our results outperform the existing deep-learning based approaches for RGB-D image complete and super-resolution. As an end-to-end approach, high fidelity RGB-D images can be generated efficiently at the rate of around 21 frames per second.

READ FULL TEXT

page 2

page 3

page 6

page 12

page 13

page 19

page 20

research
07/03/2020

Feedback Neural Network based Super-resolution of DEM for generating high fidelity features

High resolution Digital Elevation Models(DEMs) are an important requirem...
research
09/07/2022

Magnitude-image based data-consistent deep learning method for MRI super resolution

Magnetic Resonance Imaging (MRI) is important in clinic to produce high ...
research
07/23/2019

Learning High-fidelity Light Field Images From Hybrid Inputs

This paper explores the reconstruction of high-fidelity LF images (i.e.,...
research
04/14/2021

Discrete Cosine Transform Network for Guided Depth Map Super-Resolution

Guided depth super-resolution (GDSR) is a hot topic in multi-modal image...
research
05/09/2017

Signal reconstruction via operator guiding

Signal reconstruction from a sample using an orthogonal projector onto a...
research
09/22/2017

High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference

We propose a data-driven method for recovering miss-ing parts of 3D shap...
research
04/09/2020

Multi-feature super-resolution network for cloth wrinkle synthesis

Existing physical cloth simulators suffer from expensive computation and...

Please sign up or login with your details

Forgot password? Click here to reset