Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimation

11/20/2022
by   Snehal Singh Tomar, et al.
0

With an unprecedented increase in the number of agents and systems that aim to navigate the real world using visual cues and the rising impetus for 3D Vision Models, the importance of depth estimation is hard to understate. While supervised methods remain the gold standard in the domain, the copious amount of paired stereo data required to train such models makes them impractical. Most State of the Art (SOTA) works in the self-supervised and unsupervised domain employ a ResNet-based encoder architecture to predict disparity maps from a given input image which are eventually used alongside a camera pose estimator to predict depth without direct supervision. The fully convolutional nature of ResNets makes them susceptible to capturing per-pixel local information only, which is suboptimal for depth prediction. Our key insight for doing away with this bottleneck is to use Vision Transformers, which employ self-attention to capture the global contextual information present in an input image. Our model fuses per-pixel local information learned using two fully convolutional depth encoders with global contextual information learned by a transformer encoder at different scales. It does so using a mask-guided multi-stream convolution in the feature space to achieve state-of-the-art performance on most standard benchmarks.

READ FULL TEXT

page 11

page 12

page 13

research
03/31/2020

Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume

Monocular depth estimation has become one of the most studied applicatio...
research
08/06/2022

MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer

Self-supervised monocular depth estimation is an attractive solution tha...
research
02/20/2023

GlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation

In recent years, self-supervised monocular depth estimation has drawn mu...
research
12/12/2022

ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimation

The exploration of mutual-benefit cross-domains has shown great potentia...
research
12/15/2020

Semantic-Guided Representation Enhancement for Self-supervised Monocular Trained Depth Estimation

Self-supervised depth estimation has shown its great effectiveness in pr...
research
06/07/2021

Self-supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness Using On-board Videos

Self-supervised depth estimation has drawn much attention in recent year...
research
12/13/2021

GCNDepth: Self-supervised Monocular Depth Estimation based on Graph Convolutional Network

Depth estimation is a challenging task of 3D reconstruction to enhance t...

Please sign up or login with your details

Forgot password? Click here to reset