DPSNet: End-to-end Deep Plane Sweep Stereo

05/02/2019
by   Sunghoon Im, et al.
12

Multiview stereo aims to reconstruct scene depth from images acquired by a camera under arbitrary motion. Recent methods address this problem through deep learning, which can utilize semantic cues to deal with challenges such as textureless and reflective regions. In this paper, we present a convolutional neural network called DPSNet (Deep Plane Sweep Network) whose design is inspired by best practices of traditional geometry-based approaches for dense depth reconstruction. Rather than directly estimating depth and/or optical flow correspondence from image pairs as done in many previous deep learning methods, DPSNet takes a plane sweep approach that involves building a cost volume from deep features using the plane sweep algorithm, regularizing the cost volume via a context-aware cost aggregation, and regressing the dense depth map from the cost volume. The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network. Through the effective incorporation of conventional multiview stereo concepts within a deep learning framework, DPSNet achieves state-of-the-art reconstruction results on a variety of challenging datasets.

READ FULL TEXT

page 2

page 7

page 8

page 9

page 10

research
12/11/2018

DeepV2D: Video to Depth with Differentiable Structure from Motion

We propose DeepV2D, an end-to-end differentiable deep learning architect...
research
04/09/2021

CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo

Conventional stereo suffers from a fundamental trade-off between imaging...
research
11/17/2016

End-to-end Learning of Cost-Volume Aggregation for Real-time Dense Stereo

We present a new deep learning-based approach for dense stereo matching....
research
02/28/2019

SweepNet: Wide-baseline Omnidirectional Depth Estimation

Omnidirectional depth sensing has its advantage over the conventional st...
research
12/30/2022

Image-Coupled Volume Propagation for Stereo Matching

Several leading methods on public benchmarks for depth-from-stereo rely ...
research
07/17/2010

A Machine Learning Approach to Recovery of Scene Geometry from Images

Recovering the 3D structure of the scene from images yields useful infor...
research
04/19/2023

3 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset

3D dense reconstruction refers to the process of obtaining the complete ...

Please sign up or login with your details

Forgot password? Click here to reset