Learning Depth with Convolutional Spatial Propagation Network

10/04/2018
by   Xinjing Cheng, et al.
10

Depth prediction is one of the fundamental problems in computer vision. In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for various depth estimation tasks. Specifically, it is an efficient linear propagation model, in which the propagation is performed with a manner of recurrent convolutional operation, and the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). We can append this module to any output from a state-of-the-art (SOTA) depth estimation networks to improve their performances. In practice, we further extend CSPN in two aspects: 1) take sparse depth map as additional input, which is useful for the task of depth completion; 2) similar to commonly used 3D convolution operation in CNNs, we propose 3D CSPN to handle features with one additional dimension, which is effective in the task of stereo matching using 3D cost volume. For the tasks of sparse to dense, a.k.a depth completion. We experimented the proposed CPSN conjunct algorithms over the popular NYU v2 and KITTI datasets, where we show that our proposed algorithms not only produce high quality (e.g., 30 reduction in depth error), but also run faster (e.g., 2 to 5x faster) than previous SOTA spatial propagation network. We also evaluated our stereo matching algorithm on the Scene Flow and KITTI Stereo datasets, and rank 1st on both the KITTI Stereo 2012 and 2015 benchmarks, which demonstrates the effectiveness of the proposed module. The code of CSPN proposed in this work will be released at https://github.com/XinJCheng/CSPN.

READ FULL TEXT

page 2

page 5

page 6

page 7

page 8

page 11

page 12

page 15

research
08/01/2018

Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network

Depth estimation from a single image is a fundamental problem in compute...
research
10/27/2022

2T-UNET: A Two-Tower UNet with Depth Clues for Robust Stereo Depth Estimation

Stereo correspondence matching is an essential part of the multi-step st...
research
03/29/2020

Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinement

Almost all previous deep learning-based multi-view stereo (MVS) approach...
research
03/23/2018

Pyramid Stereo Matching Network

Recent work has shown that depth estimation from a stereo pair of images...
research
07/11/2019

Efficient Semantic Scene Completion Network with Spatial Group Convolution

We introduce Spatial Group Convolution (SGC) for accelerating the comput...
research
09/14/2022

FCDSN-DC: An Accurate and Lightweight Convolutional Neural Network for Stereo Estimation with Depth Completion

We propose an accurate and lightweight convolutional neural network for ...
research
09/18/2017

Minimal Effort Back Propagation for Convolutional Neural Networks

As traditional neural network consumes a significant amount of computing...

Please sign up or login with your details

Forgot password? Click here to reset