DeepAI AI Chat
Log In Sign Up

Joint Learning of Frequency and Spatial Domains for Dense Predictions

by   Shaocheng Jia, et al.
The Hong Kong Polytechnic University

Current artificial neural networks mainly conduct the learning process in the spatial domain but neglect the frequency domain learning. However, the learning course performed in the frequency domain can be more efficient than that in the spatial domain. In this paper, we fully explore frequency domain learning and propose a joint learning paradigm of frequency and spatial domains. This paradigm can take full advantage of the preponderances of frequency learning and spatial learning; specifically, frequency and spatial domain learning can effectively capture global and local information, respectively. Exhaustive experiments on two dense prediction tasks, i.e., self-supervised depth estimation and semantic segmentation, demonstrate that the proposed joint learning paradigm can 1) achieve performance competitive to those of state-of-the-art methods in both depth estimation and semantic segmentation tasks, even without pretraining; and 2) significantly reduce the number of parameters compared to other state-of-the-art methods, which provides more chance to develop real-world applications. We hope that the proposed method can encourage more research in cross-domain learning.


page 12

page 15

page 16

page 18

page 19

page 20

page 21

page 22


Self-Supervised Monocular Depth Estimation with Internal Feature Fusion

Self-supervised learning for depth estimation uses geometry in image seq...

Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth

Multi-task learning (MTL) paradigm focuses on jointly learning two or mo...

False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimation

State-of-the-art deep neural networks demonstrate outstanding performanc...

Frequency-domain Learning for Volumetric-based 3D Data Perception

Frequency-domain learning draws attention due to its superior tradeoff b...

Context-Aware Domain Adaptation in Semantic Segmentation

In this paper, we consider the problem of unsupervised domain adaptation...

Learning in the Frequency Domain

Deep neural networks have achieved remarkable success in computer vision...

Lifelong-MonoDepth: Lifelong Learning for Multi-Domain Monocular Metric Depth Estimation

In recent years, monocular depth estimation (MDE) has gained significant...