SideRT: A Real-time Pure Transformer Architecture for Single Image Depth Estimation

04/29/2022
by   Chang Shu, et al.
5

Since context modeling is critical for estimating depth from a single image, researchers put tremendous effort into obtaining global context. Many global manipulations are designed for traditional CNN-based architectures to overcome the locality of convolutions. Attention mechanisms or transformers originally designed for capturing long-range dependencies might be a better choice, but usually complicates architectures and could lead to a decrease in inference speed. In this work, we propose a pure transformer architecture called SideRT that can attain excellent predictions in real-time. In order to capture better global context, Cross-Scale Attention (CSA) and Multi-Scale Refinement (MSR) modules are designed to work collaboratively to fuse features of different scales efficiently. CSA modules focus on fusing features of high semantic similarities, while MSR modules aim to fuse features at corresponding positions. These two modules contain a few learnable parameters without convolutions, based on which a lightweight yet effective model is built. This architecture achieves state-of-the-art performances in real-time (51.3 FPS) and becomes much faster with a reasonable performance drop on a smaller backbone Swin-T (83.1 FPS). Furthermore, its performance surpasses the previous state-of-the-art by a large margin, improving AbsRel metric 6.9 9.7 transformer-based networks can attain state-of-the-art performance in real-time in the single image depth estimation field. Code will be made available soon.

READ FULL TEXT

page 3

page 5

page 6

research
11/23/2022

Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation

Self-supervised monocular depth estimation that does not require ground-...
research
02/06/2022

GLPanoDepth: Global-to-Local Panoramic Depth Estimation

In this paper, we propose a learning-based method for predicting dense d...
research
04/25/2023

CompletionFormer: Depth Completion with Convolutions and Vision Transformers

Given sparse depths and the corresponding RGB images, depth completion a...
research
06/15/2023

Motion Perceiver: Real-Time Occupancy Forecasting for Embedded Systems

This work introduces a flexible architecture for real-time occupancy for...
research
03/13/2023

DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes

Varicolored haze caused by chromatic casts poses haze removal and depth ...
research
03/06/2023

DwinFormer: Dual Window Transformers for End-to-End Monocular Depth Estimation

Depth estimation from a single image is of paramount importance in the r...
research
04/15/2021

Geometry-Free View Synthesis: Transformers and no 3D Priors

Is a geometric model required to synthesize novel views from a single im...

Please sign up or login with your details

Forgot password? Click here to reset