TW-SMNet: Deep Multitask Learning of Tele-Wide Stereo Matching

by   Mostafa El-Khamy, et al.

In this paper, we introduce the problem of estimating the real world depth of elements in a scene captured by two cameras with different field of views, where the first field of view (FOV) is a Wide FOV (WFOV) captured by a wide angle lens, and the second FOV is contained in the first FOV and is captured by a tele zoom lens. We refer to the problem of estimating the inverse depth for the union of FOVs, while leveraging the stereo information in the overlapping FOV, as Tele-Wide Stereo Matching (TW-SM). We propose different deep learning solutions to the TW-SM problem. Since the disparity is proportional to the inverse depth, we train stereo matching disparity estimation (SMDE) networks to estimate the disparity for the union WFOV. We further propose an end-to-end deep multitask tele-wide stereo matching neural network (MT-TW-SMNet), which simultaneously learns the SMDE task for the overlapped Tele FOV and the single image inverse depth estimation (SIDE) task for the WFOV. Moreover, we design multiple methods for the fusion of the SMDE and SIDE networks. We evaluate the performance of TW-SM on the popular KITTI and SceneFlow stereo datasets, and demonstrate its practicality by synthesizing the Bokeh effect on the WFOV from a tele-wide stereo image pair.


page 2

page 3

page 13

page 15

page 16

page 17

page 19


Obtaining Depth Maps From Color Images By Region Based Stereo Matching Algorithms

In the paper, region based stereo matching algorithms are developed for ...

DSR: Direct Self-rectification for Uncalibrated Dual-lens Cameras

With the developments of dual-lens camera modules,depth information repr...

Hyperspectral Light Field Stereo Matching

In this paper, we describe how scene depth can be extracted using a hype...

SBEVNet: End-to-End Deep Stereo Layout Estimation

Accurate layout estimation is crucial for planning and navigation in rob...

Uncertainty Estimation for End-To-End Learned Dense Stereo Matching via Probabilistic Deep Learning

Motivated by the need to identify erroneous disparity assignments, vario...

Bayesian dense inverse searching algorithm for real-time stereo matching in minimally invasive surgery

This paper reports a CPU-level real-time stereo matching method for surg...

360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume

Recently, end-to-end trainable deep neural networks have significantly i...

Please sign up or login with your details

Forgot password? Click here to reset