Generalized Binary Search Network for Highly-Efficient Multi-View Stereo

by   Zhenxing Mi, et al.
The Hong Kong University of Science and Technology

Multi-view Stereo (MVS) with known camera parameters is essentially a 1D search problem within a valid depth range. Recent deep learning-based MVS methods typically densely sample depth hypotheses in the depth range, and then construct prohibitively memory-consuming 3D cost volumes for depth prediction. Although coarse-to-fine sampling strategies alleviate this overhead issue to a certain extent, the efficiency of MVS is still an open challenge. In this work, we propose a novel method for highly efficient MVS that remarkably decreases the memory footprint, meanwhile clearly advancing state-of-the-art depth prediction performance. We investigate what a search strategy can be reasonably optimal for MVS taking into account of both efficiency and effectiveness. We first formulate MVS as a binary search problem, and accordingly propose a generalized binary search network for MVS. Specifically, in each step, the depth range is split into 2 bins with extra 1 error tolerance bin on both sides. A classification is performed to identify which bin contains the true depth. We also design three mechanisms to respectively handle classification errors, deal with out-of-range samples and decrease the training memory. The new formulation makes our method only sample a very small number of depth hypotheses in each step, which is highly memory efficient, and also greatly facilitates quick training convergence. Experiments on competitive benchmarks show that our method achieves state-of-the-art accuracy with much less memory. Particularly, our method obtains an overall score of 0.289 on DTU dataset and tops the first place on challenging Tanks and Temples advanced dataset among all the learning-based methods. The trained models and code will be released at


page 3

page 5

page 6

page 10

page 11

page 12

page 13

page 14


DDR-Net: Learning Multi-Stage Multi-View Stereo With Dynamic Depth Range

To obtain high-resolution depth maps, some previous learning-based multi...

Cost Volume Pyramid Network with Multi-strategies Range Searching for Multi-view Stereo

Multi-view stereo is an important research task in computer vision while...

RayMVSNet: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo

Learning-based multi-view stereo (MVS) has by far centered around 3D con...

IB-MVS: An Iterative Algorithm for Deep Multi-View Stereo based on Binary Decisions

We present a novel deep-learning-based method for Multi-View Stereo. Our...

ARAI-MVSNet: A multi-view stereo depth estimation network with adaptive depth range and depth interval

Multi-View Stereo (MVS) is a fundamental problem in geometric computer v...

MVSTER: Epipolar Transformer for Efficient Multi-View Stereo

Learning-based Multi-View Stereo (MVS) methods warp source images into t...

DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion

We propose an online multi-view depth prediction approach on posed video...

Please sign up or login with your details

Forgot password? Click here to reset