360^∘ High-Resolution Depth Estimation via Uncertainty-aware Structural Knowledge Transfer

by   Zidong Cao, et al.
The Hong Kong University of Science and Technology

Recently, omnidirectional images (ODIs) have become increasingly popular; however, their angular resolution tends to be lower than that of perspective images.This leads to degraded structural details such as edges, causing difficulty in learning 3D scene understanding tasks, especially monocular depth estimation. Existing methods typically leverage high-resolution (HR) ODI as the input, so as to recover the structural details via fully-supervised learning. However, the HR depth ground truth (GT) maps may be arduous or expensive to be collected due to resource-constrained devices in practice. Therefore, in this paper, we explore for the first time to estimate the HR omnidirectional depth directly from a low-resolution (LR) ODI, when no HR depth GT map is available. Our key idea is to transfer the scene structural knowledge from the readily available HR image modality and the corresponding LR depth maps to achieve the goal of HR depth estimation without extra inference cost. Specifically, we introduce ODI super-resolution (SR) as an auxiliary task and train both tasks collaboratively in a weakly supervised manner to boost the performance of HR depth estimation. The ODI SR task takes an LR ODI as the input to predict an HR image, enabling us to extract the scene structural knowledge via uncertainty estimation. Buttressed by this, a scene structural knowledge transfer (SSKT) module is proposed with two key components. First, we employ a cylindrical implicit interpolation function (CIIF) to learn cylindrical neural interpolation weights for feature up-sampling and share the parameters of CIIFs between the two tasks. Then, we propose a feature distillation (FD) loss that provides extra structural regularization to help the HR depth estimation task learn more scene structural knowledge.


page 1

page 4

page 7

page 8

page 9


Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution

Existing color-guided depth super-resolution (DSR) approaches require pa...

High Quality Monocular Depth Estimation via Transfer Learning

Accurate depth estimation from images is a fundamental task in many appl...

SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation

Recent techniques in self-supervised monocular depth estimation are appr...

Monocular Depth Estimation: A Survey

Monocular depth estimation is often described as an ill-posed and inhere...

Unsupervised High-Resolution Depth Learning From Videos With Dual Networks

Unsupervised depth learning takes the appearance difference between a ta...

PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing

Depth estimation and scene parsing are two particularly important tasks ...

Please sign up or login with your details

Forgot password? Click here to reset