Self-supervised Learning of Occlusion Aware Flow Guided 3D Geometry Perception with Adaptive Cross Weighted Loss from Monocular Videos

08/09/2021
by   Jiaojiao Fang, et al.
3

Self-supervised deep learning-based 3D scene understanding methods can overcome the difficulty of acquiring the densely labeled ground-truth and have made a lot of advances. However, occlusions and moving objects are still some of the major limitations. In this paper, we explore the learnable occlusion aware optical flow guided self-supervised depth and camera pose estimation by an adaptive cross weighted loss to address the above limitations. Firstly, we explore to train the learnable occlusion mask fused optical flow network by an occlusion-aware photometric loss with the temporally supplemental information and backward-forward consistency of adjacent views. And then, we design an adaptive cross-weighted loss between the depth-pose and optical flow loss of the geometric and photometric error to distinguish the moving objects which violate the static scene assumption. Our method shows promising results on KITTI, Make3D, and Cityscapes datasets under multiple tasks. We also show good generalization ability under a variety of challenging scenarios.

READ FULL TEXT

page 1

page 2

page 4

page 5

research
07/12/2019

Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Camera

We present GLNet, a self-supervised framework for learning depth, optica...
research
04/08/2020

Self-Supervised Monocular Scene Flow Estimation

Scene flow estimation has been receiving increasing attention for 3D env...
research
06/27/2019

DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints

This paper presents an self-supervised deep learning network for monocul...
research
12/12/2022

CbwLoss: Constrained Bidirectional Weighted Loss for Self-supervised Learning of Depth and Pose

Photometric differences are widely used as supervision signals to train ...
research
08/14/2023

DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume

Self-supervised monocular depth estimation methods typically rely on the...
research
06/30/2022

Self-SuperFlow: Self-supervised Scene Flow Prediction in Stereo Sequences

In recent years, deep neural networks showed their exceeding capabilitie...
research
03/29/2021

Generalizing to the Open World: Deep Visual Odometry with Online Adaptation

Despite learning-based visual odometry (VO) has shown impressive results...

Please sign up or login with your details

Forgot password? Click here to reset