MSMD-Net: Deep Stereo Matching with Multi-scale and Multi-dimension Cost Volume

06/23/2020
by   Zhelun Shen, et al.
0

Deep end-to-end learning based stereo matching methods have achieved great success as witnessed by the leaderboards across different benchmarking datasets (KITTI, Middlebury, ETH3D, etc), where the cost volume representation is an indispensable step to the success. However, most existing work only employs a single cost volume, which cannot fully exploit the multi-scale cues in stereo matching and provide guidance for disparity refinement. What's more, the single cost volume representation also limits the disparity range and the resolution of the disparity estimation. In this paper, we propose MSMD-Net (Multi-Scale and Multi-Dimension) to construct multi-scale and multi-dimension cost volume. At the multi-scale level, we generate four 4D combination volumes at different scales and integrate them in 3D cost aggregation to predict an initial disparity estimation. At the multi-dimension level, we construct a 3D warped correlation volume and use it to refine the initial disparity map with residual learning. These two dimensional cost volumes are complementary to each other and can boost the performance of disparity estimation. Additionally, we propose a switch training strategy to further improve the accuracy of disparity estimation, where we switch two kinds of different activation functions to alleviate the overfitting issue in the pre-training process. Our proposed method was evaluated on several benchmark datasets and ranked first on KITTI 2012 leaderboard and second on KITTI 2015 leaderboard as of June 23.The code of MSMD-Net is available at https://github.com/gallenszl/MSMD-Net.

READ FULL TEXT

page 7

page 8

research
04/25/2019

MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching

Disparity prediction from stereo images is essential to computer vision ...
research
12/04/2017

Learning Deep Correspondence through Prior and Posterior Feature Constancy

Stereo matching algorithms usually consist of four steps, including matc...
research
03/10/2019

Group-wise Correlation Stereo Network

Stereo matching estimates the disparity between a rectified image pair, ...
research
04/09/2021

CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching

Recently, the ever-increasing capacity of large-scale annotated datasets...
research
03/24/2020

FADNet: A Fast and Accurate Network for Disparity Estimation

Deep neural networks (DNNs) have achieved great success in the area of c...
research
06/05/2020

Content-Aware Inter-Scale Cost Aggregation for Stereo Matching

Cost aggregation is a key component of stereo matching for high-quality ...
research
05/25/2021

SRH-Net: Stacked Recurrent Hourglass Network for Stereo Matching

The cost aggregation strategy shows a crucial role in learning-based ste...

Please sign up or login with your details

Forgot password? Click here to reset