Transform Once: Efficient Operator Learning in Frequency Domain

by   Michael Poli, et al.

Spectral analysis provides one of the most effective paradigms for information-preserving dimensionality reduction, as simple descriptions of naturally occurring signals are often obtained via few terms of periodic basis functions. In this work, we study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time: frequency-domain models (FDMs). Existing FDMs are based on complex-valued transforms i.e. Fourier Transforms (FT), and layers that perform computation on the spectrum and input data separately. This design introduces considerable computational overhead: for each layer, a forward and inverse FT. Instead, this work introduces a blueprint for frequency domain learning through a single transform: transform once (T1). To enable efficient, direct learning in the frequency domain we derive a variance-preserving weight initialization scheme and investigate methods for frequency selection in reduced-order FDMs. Our results noticeably streamline the design process of FDMs, pruning redundant transforms, and leading to speedups of 3x to 10x that increase with data resolution and model size. We perform extensive experiments on learning the solution operator of spatio-temporal dynamics, including incompressible Navier-Stokes, turbulent flows around airfoils and high-resolution video of smoke. T1 models improve on the test performance of FDMs while requiring significantly less computation (5 hours instead of 32 for our large-scale experiment), with over 20


page 6

page 8

page 10

page 19

page 21

page 23

page 25


Spectral Convolution Networks

Previous research has shown that computation of convolution in the frequ...

Clifford Fourier-Mellin transform with two real square roots of -1 in Cl(p,q), p+q=2

We describe a non-commutative generalization of the complex Fourier-Mell...

PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Prediction

In this paper, we investigate the challenge of spatio-temporal video pre...

Spectral Representations for Convolutional Neural Networks

Discrete Fourier transforms provide a significant speedup in the computa...

Orthogonal and Non-Orthogonal Signal Representations Using New Transformation Matrices Having NPM Structure

In this paper, we introduce two types of real-valued sums known as Compl...

ESResNe(X)t-fbsp: Learning Robust Time-Frequency Transformation of Audio

Environmental Sound Classification (ESC) is a rapidly evolving field tha...

Star DGT: a Robust Gabor Transform for Speech Denoising

In this paper, we address the speech denoising problem, where white Gaus...

Please sign up or login with your details

Forgot password? Click here to reset