DRO: Deep Recurrent Optimizer for Structure-from-Motion

03/24/2021
by   Xiaodong Gu, et al.
23

There are increasing interests of studying the structure-from-motion (SfM) problem with machine learning techniques. While earlier methods directly learn a mapping from images to depth maps and camera poses, more recent works enforce multi-view geometry through optimization embed in the learning framework. This paper presents a novel optimization method based on recurrent neural networks to further exploit the potential of neural networks in SfM. Our neural optimizer alternatively updates the depth and camera poses through iterations to minimize a feature-metric cost. Two gated recurrent units are designed to trace the historical information during the iterations. Our network works as a zeroth-order optimizer, where the computation and memory expensive cost volume or gradients are avoided. Experiments demonstrate that our recurrent optimizer effectively reduces the feature-metric cost while refining the depth and poses. Our method outperforms previous methods and is more efficient in computation and memory consumption than cost-volume-based methods. The code of our method will be made public.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 13

page 14

research
02/27/2019

Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference

Deep learning has recently demonstrated its excellent performance for mu...
research
04/04/2022

RayMVSNet: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo

Learning-based multi-view stereo (MVS) has by far centered around 3D con...
research
10/21/2019

Learning to Learn by Zeroth-Order Oracle

In the learning to learn (L2L) framework, we cast the design of optimiza...
research
12/14/2022

RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging

This paper proposes a deep recurrent Rotation Averaging Graph Optimizer ...
research
05/28/2022

RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo

In this paper, we present a learning-based approach for multi-view stere...
research
03/03/2022

NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation

Estimating the accurate depth from a single image is challenging since i...
research
03/27/2019

A Multi Hidden Recurrent Neural Network with a Modified Grey Wolf Optimizer

Identifying university students' weaknesses results in better learning a...

Please sign up or login with your details

Forgot password? Click here to reset