Monocular Depth Estimation Using Cues Inspired by Biological Vision Systems

04/21/2022
by   Dylan Auty, et al.
0

Monocular depth estimation (MDE) aims to transform an RGB image of a scene into a pixelwise depth map from the same camera view. It is fundamentally ill-posed due to missing information: any single image can have been taken from many possible 3D scenes. Part of the MDE task is, therefore, to learn which visual cues in the image can be used for depth estimation, and how. With training data limited by cost of annotation or network capacity limited by computational power, this is challenging. In this work we demonstrate that explicitly injecting visual cue information into the model is beneficial for depth estimation. Following research into biological vision systems, we focus on semantic information and prior knowledge of object sizes and their relations, to emulate the biological cues of relative size, familiar size, and absolute size. We use state-of-the-art semantic and instance segmentation models to provide external information, and exploit language embeddings to encode relational information between classes. We also provide a prior on the average real-world size of objects. This external information overcomes the limitation in data availability, and ensures that the limited capacity of a given network is focused on known-helpful cues, therefore improving performance. We experimentally validate our hypothesis and evaluate the proposed model on the widely used NYUD2 indoor depth estimation benchmark. The results show improvements in depth prediction when the semantic information, size prior and instance size are explicitly provided along with the RGB images, and our method can be easily adapted to any depth estimation system.

READ FULL TEXT

page 1

page 2

page 3

research
04/25/2023

Depth-Relative Self Attention for Monocular Depth Estimation

Monocular depth estimation is very challenging because clues to the exac...
research
10/09/2018

Geometry meets semantics for semi-supervised monocular depth estimation

Depth estimation from a single image represents a very exciting challeng...
research
03/21/2018

Monocular Depth Estimation by Learning from Heterogeneous Datasets

Depth estimation provides essential information to perform autonomous dr...
research
10/06/2020

Parallax Motion Effect Generation Through Instance Segmentation And Depth Estimation

Stereo vision is a growing topic in computer vision due to the innumerab...
research
08/09/2022

The Relative Importance of Depth Cues and Semantic Edges for Indoor Mobility Using Simulated Prosthetic Vision in Immersive Virtual Reality

Visual neuroprostheses (bionic eyes) have the potential to treat degener...
research
02/10/2021

Exploiting Depth Information for Wildlife Monitoring

Camera traps are a proven tool in biology and specifically biodiversity ...
research
11/30/2022

ObjCAViT: Improving Monocular Depth Estimation Using Natural Language Models And Image-Object Cross-Attention

While monocular depth estimation (MDE) is an important problem in comput...

Please sign up or login with your details

Forgot password? Click here to reset