Does Monocular Depth Estimation Provide Better Pre-training than Classification for Semantic Segmentation?

03/26/2022
by   Dong Lao, et al.
0

Training a deep neural network for semantic segmentation is labor-intensive, so it is common to pre-train it for a different task, and then fine-tune it with a small annotated dataset. State-of-the-art methods use image classification for pre-training, which introduces uncontrolled biases. We test the hypothesis that depth estimation from unlabeled videos may provide better pre-training. Despite the absence of any semantic information, we argue that estimating scene geometry is closer to the task of semantic segmentation than classifying whole images into semantic classes. Since analytical validation is intractable, we test the hypothesis empirically by introducing a pre-training scheme that yields an improvement of 5.7 classification-based pre-training. While annotation is not needed for pre-training, it is needed for testing the hypothesis. We use the KITTI (outdoor) and NYU-V2 (indoor) benchmarks to that end, and provide an extensive discussion of the benefits and limitations of the proposed scheme in relation to existing unsupervised, self-supervised, and semi-supervised pre-training protocols.

READ FULL TEXT
research
02/06/2020

RGB-based Semantic Segmentation Using Self-Supervised Depth Pre-Training

Although well-known large-scale datasets, such as ImageNet, have driven ...
research
12/19/2020

Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation

Training deep networks for semantic segmentation requires large amounts ...
research
07/26/2023

Pre-Training with Diffusion models for Dental Radiography segmentation

Medical radiography segmentation, and specifically dental radiography, i...
research
02/14/2022

COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets

Transfer learning is a proven technique in 2D computer vision to leverag...
research
07/11/2022

Demystifying Unsupervised Semantic Correspondence Estimation

We explore semantic correspondence estimation through the lens of unsupe...
research
06/08/2022

Delving into the Pre-training Paradigm of Monocular 3D Object Detection

The labels of monocular 3D object detection (M3OD) are expensive to obta...
research
04/16/2022

Language-Grounded Indoor 3D Semantic Segmentation in the Wild

Recent advances in 3D semantic segmentation with deep neural networks ha...

Please sign up or login with your details

Forgot password? Click here to reset