Frequency-domain Learning for Volumetric-based 3D Data Perception

02/16/2023
by   Zifan Yu, et al.
0

Frequency-domain learning draws attention due to its superior tradeoff between inference accuracy and input data size. Frequency-domain learning in 2D computer vision tasks has shown that 2D convolutional neural networks (CNN) have a stationary spectral bias towards low-frequency channels so that high-frequency channels can be pruned with no or little accuracy degradation. However, frequency-domain learning has not been studied in the context of 3D CNNs with 3D volumetric data. In this paper, we study frequency-domain learning for volumetric-based 3D data perception to reveal the spectral bias and the accuracy-input-data-size tradeoff of 3D CNNs. Our study finds that 3D CNNs are sensitive to a limited number of critical frequency channels, especially low-frequency channels. Experiment results show that frequency-domain learning can significantly reduce the size of volumetric-based 3D inputs (based on spectral bias) while achieving comparable accuracy with conventional spatial-domain learning approaches. Specifically, frequency-domain learning is able to reduce the input data size by 98 limiting the average accuracy drop within 2 semantic segmentation with a 1.48 limiting the mean-class IoU loss within 1.55 higher-resolution 3D data (i.e., 2x of the original image in the spatial domain), frequency-domain learning improves the mean-class accuracy and mean-class IoU by 3.04 data size reduction in 3D point cloud semantic segmentation.

READ FULL TEXT

page 8

page 9

page 12

research
02/27/2020

Learning in the Frequency Domain

Deep neural networks have achieved remarkable success in computer vision...
research
12/28/2020

Spectral Analysis for Semantic Segmentation with Applications on Feature Truncation and Weak Annotation

The current neural networks for semantic segmentation usually predict th...
research
02/18/2022

Joint Learning of Frequency and Spatial Domains for Dense Predictions

Current artificial neural networks mainly conduct the learning process i...
research
05/06/2022

Investigating and Explaining the Frequency Bias in Image Classification

CNNs exhibit many behaviors different from humans, one of which is the c...
research
11/27/2019

Exploring Frequency Domain Interpretation of Convolutional Neural Networks

Many existing interpretation methods of convolutional neural networks (C...
research
08/09/2021

The Weighted Average Illusion: Biases in Perceived Mean Position in Scatterplots

Scatterplots can encode a third dimension by using additional channels l...
research
07/18/2020

Volumetric Transformer Networks

Existing techniques to encode spatial invariance within deep convolution...

Please sign up or login with your details

Forgot password? Click here to reset