DeepAI AI Chat
Log In Sign Up

Self-Supervised Multi-Frame Monocular Scene Flow

by   Junhwa Hur, et al.

Estimating 3D scene flow from a sequence of monocular images has been gaining increased attention due to the simple, economical capture setup. Owing to the severe ill-posedness of the problem, the accuracy of current methods has been limited, especially that of efficient, real-time approaches. In this paper, we introduce a multi-frame monocular scene flow network based on self-supervised learning, improving the accuracy over previous networks while retaining real-time efficiency. Based on an advanced two-frame baseline with a split-decoder design, we propose (i) a multi-frame model using a triple frame input and convolutional LSTM connections, (ii) an occlusion-aware census loss for better accuracy, and (iii) a gradient detaching strategy to improve training stability. On the KITTI dataset, we observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.


page 1

page 4

page 8

page 12

page 15

page 16

page 17

page 18


RAFT-MSF: Self-Supervised Monocular Scene Flow using Recurrent Optimizer

Learning scene flow from a monocular camera still remains a challenging ...

Self-Supervised Monocular Scene Flow Estimation

Scene flow estimation has been receiving increasing attention for 3D env...

Monocular Depth Estimation with Self-supervised Instance Adaptation

Recent advances in self-supervised learning havedemonstrated that it is ...

Unsupervised Hebbian Learning on Point Sets in StarCraft II

Learning the evolution of real-time strategy (RTS) game is a challenging...

Cascade Network for Self-Supervised Monocular Depth Estimation

It is a classical compute vision problem to obtain real scene depth maps...

A Deep Temporal Fusion Framework for Scene Flow Using a Learnable Motion Model and Occlusions

Motion estimation is one of the core challenges in computer vision. With...

Code Repositories


Self-Supervised Multi-Frame Monocular Scene Flow (CVPR 2021)

view repo