Log In Sign Up

Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance

by   Marvin Klingner, et al.

Self-supervised monocular depth estimation presents a powerful method to obtain 3D scene information from single camera images, which is trainable on arbitrary image sequences without requiring depth labels, e.g., from a LiDAR sensor. In this work we present a new self-supervised semantically-guided depth estimation (SGDepth) method to deal with moving dynamic-class (DC) objects, such as moving cars and pedestrians, which violate the static-world assumptions typically made during training of such models. Specifically, we propose (i) mutually beneficial cross-domain training of (supervised) semantic segmentation and self-supervised depth estimation with task-specific network heads, (ii) a semantic masking scheme providing guidance to prevent moving DC objects from contaminating the photometric loss, and (iii) a detection method for frames with non-moving DC objects, from which the depth of DC objects can be learned. We demonstrate the performance of our method on several benchmarks, in particular on the Eigen split, where we exceed all baselines without test-time refinement in all measures.


page 8

page 13

page 14

page 24

page 25


Bootstrapped Self-Supervised Training with Monocular Video for Semantic Segmentation and Depth Estimation

For a robot deployed in the world, it is desirable to have the ability o...

Dyna-DM: Dynamic Object-aware Self-supervised Monocular Depth Maps

Self-supervised monocular depth estimation has been a subject of intense...

Self-Supervised Depth Learning for Urban Scene Understanding

As an agent moves through the world, the apparent motion of scene elemen...

D^2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video

Given a monocular video, segmenting and decoupling dynamic objects while...

Instance-aware multi-object self-supervision for monocular depth prediction

This paper proposes a self-supervised monocular image-to-depth predictio...

Self-Supervised Monocular Depth Estimation: Solving the Edge-Fattening Problem

Self-supervised monocular depth estimation (MDE) models universally suff...