DeepAI
Log In Sign Up

Rethinking FUN: Frequency-Domain Utilization Networks

12/06/2020
by   Kfir Goldberg, et al.
0

The search for efficient neural network architectures has gained much focus in recent years, where modern architectures focus not only on accuracy but also on inference time and model size. Here, we present FUN, a family of novel Frequency-domain Utilization Networks. These networks utilize the inherent efficiency of the frequency-domain by working directly in that domain, represented with the Discrete Cosine Transform. Using modern techniques and building blocks such as compound-scaling and inverted-residual layers we generate a set of such networks allowing one to balance between size, latency and accuracy while outperforming competing RGB-based models. Extensive evaluations verifies that our networks present strong alternatives to previous approaches. Moreover, we show that working in frequency domain allows for dynamic compression of the input at inference time without any explicit change to the architecture.

READ FULL TEXT

page 3

page 8

11/21/2019

Band-limited Training and Inference for Convolutional Neural Networks

The convolutional layers are core building blocks of neural network arch...
09/13/2021

Real-Time EMG Signal Classification via Recurrent Neural Networks

Real-time classification of Electromyography signals is the most challen...
06/18/2020

Image classification in frequency domain with 2SReLU: a second harmonics superposition activation function

Deep Convolutional Neural Networks are able to identify complex patterns...
11/21/2019

Fast Sparse ConvNets

Historically, the pursuit of efficient inference has been one of the dri...
01/18/2020

Harmonic Convolutional Networks based on Discrete Cosine Transform

Convolutional neural networks (CNNs) learn filters in order to capture l...
10/05/2020

DCT-SNN: Using DCT to Distribute Spatial Information over Time for Learning Low-Latency Spiking Neural Networks

Spiking Neural Networks (SNNs) offer a promising alternative to traditio...
06/02/2017

Dynamic Steerable Blocks in Deep Residual Networks

Filters in convolutional networks are typically parameterized in a pixel...