RayMVSNet: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo

04/04/2022
by   Junhua Xi, et al.
0

Learning-based multi-view stereo (MVS) has by far centered around 3D convolution on cost volumes. Due to the high computation and memory consumption of 3D CNN, the resolution of output depth is often considerably limited. Different from most existing works dedicated to adaptive refinement of cost volumes, we opt to directly optimize the depth value along each camera ray, mimicking the range (depth) finding of a laser scanner. This reduces the MVS problem to ray-based depth optimization which is much more light-weight than full cost volume optimization. In particular, we propose RayMVSNet which learns sequential prediction of a 1D implicit field along each camera ray with the zero-crossing point indicating scene depth. This sequential modeling, conducted based on transformer features, essentially learns the epipolar line search in traditional multi-view stereo. We also devise a multi-task learning for better optimization convergence and depth accuracy. Our method ranks top on both the DTU and the Tanks & Temples datasets over all previous learning-based methods, achieving overall reconstruction score of 0.33mm on DTU and f-score of 59.48 on Tanks Temples.

READ FULL TEXT

page 3

page 4

page 5

page 6

page 8

research
07/25/2022

Cost Volume Pyramid Network with Multi-strategies Range Searching for Multi-view Stereo

Multi-view stereo is an important research task in computer vision while...
research
04/15/2022

MVSTER: Epipolar Transformer for Efficient Multi-View Stereo

Learning-based Multi-View Stereo (MVS) methods warp source images into t...
research
03/24/2021

DRO: Deep Recurrent Optimizer for Structure-from-Motion

There are increasing interests of studying the structure-from-motion (Sf...
research
12/13/2022

DELS-MVS: Deep Epipolar Line Search for Multi-View Stereo

We propose a novel approach for deep learning-based Multi-View Stereo (M...
research
12/04/2021

Generalized Binary Search Network for Highly-Efficient Multi-View Stereo

Multi-view Stereo (MVS) with known camera parameters is essentially a 1D...
research
12/03/2020

DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion

We propose an online multi-view depth prediction approach on posed video...
research
03/15/2023

Implicit Ray-Transformers for Multi-view Remote Sensing Image Segmentation

The mainstream CNN-based remote sensing (RS) image semantic segmentation...

Please sign up or login with your details

Forgot password? Click here to reset