DeepAI
Log In Sign Up

Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation

12/19/2020
by   Lukas Hoyer, et al.
9

Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process. To address this issue, we present a framework for semi-supervised semantic segmentation, which is enhanced by self-supervised monocular depth estimation from unlabeled images. In particular, we propose three key contributions: (1) We transfer knowledge from features learned during self-supervised depth estimation to semantic segmentation, (2) we implement a strong data augmentation by blending images and labels using the structure of the scene, and (3) we utilize the depth feature diversity as well as the level of difficulty of learning depth in a student-teacher framework to select the most useful samples to be annotated for semantic segmentation. We validate the proposed model on the Cityscapes dataset, where all three modules demonstrate significant performance gains, and we achieve state-of-the-art results for semi-supervised semantic segmentation. The implementation is available at https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.

READ FULL TEXT

page 4

page 5

page 6

page 8

page 13

page 14

page 15

03/19/2021

Bootstrapped Self-Supervised Training with Monocular Video for Semantic Segmentation and Depth Estimation

For a robot deployed in the world, it is desirable to have the ability o...
06/21/2022

Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth

Multi-task learning (MTL) paradigm focuses on jointly learning two or mo...
07/07/2022

False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimation

State-of-the-art deep neural networks demonstrate outstanding performanc...
11/11/2020

A comparative study of semi- and self-supervised semantic segmentation of biomedical microscopy data

In recent years, Convolutional Neural Networks (CNNs) have become the st...
06/26/2021

OffRoadTranSeg: Semi-Supervised Segmentation using Transformers on OffRoad environments

We present OffRoadTranSeg, the first end-to-end framework for semi-super...
06/15/2020

Self-Supervised Domain Mismatch Estimation for Autonomous Perception

Autonomous driving requires self awareness of its perception functions. ...
02/18/2022

Joint Learning of Frequency and Spatial Domains for Dense Predictions

Current artificial neural networks mainly conduct the learning process i...

Code Repositories

improving_segmentation_with_selfsupervised_depth

An implementation of our work "Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation"


view repo