LinearConv: Regenerating Redundancy in Convolution Filters as Linear Combinations for Parameter Reduction

07/26/2019
by   Kumara Kahatapitiya, et al.
0

Convolutional Neural Networks (CNNs) show state-of-the-art performance in computer vision tasks. However, convolutional layers of CNNs are known to learn redundant features, still not being efficient in memory requirement. In this work, we explore the redundancy of learned features in the form of correlation between convolutional filters, and propose a novel layer to reproduce it efficiently. The proposed "LinearConv" layer generates a portion of convolutional filters as a learnable linear combination of the rest of the filters and introduces a correlation-based regularization to achieve flexibility and control over the correlation between filters, and the number of parameters, in turn. This is developed as a plug-in layer to conveniently replace a conventional convolutional layer without any modification required in the network architecture. Our experiments verify that the LinearConv-based models are able to achieve a performance on-par with counterparts with up to 50 time.

READ FULL TEXT
research
08/23/2019

Learning Filter Basis for Convolutional Neural Network Compression

Convolutional neural networks (CNNs) based solutions have achieved state...
research
01/23/2018

Learning to Prune Filters in Convolutional Neural Networks

Many state-of-the-art computer vision algorithms use large scale convolu...
research
07/03/2018

Stochastic Channel Decorrelation Network and Its Application to Visual Tracking

Deep convolutional neural networks (CNNs) have dominated many computer v...
research
02/27/2019

Modulated binary cliquenet

Although Convolutional Neural Networks (CNNs) achieve effectiveness in v...
research
05/24/2023

Focus Your Attention (with Adaptive IIR Filters)

We present a new layer in which dynamic (i.e.,input-dependent) Infinite ...
research
06/22/2020

DO-Conv: Depthwise Over-parameterized Convolutional Layer

Convolutional layers are the core building blocks of Convolutional Neura...
research
10/29/2021

Gabor filter incorporated CNN for compression

Convolutional neural networks (CNNs) are remarkably successful in many c...

Please sign up or login with your details

Forgot password? Click here to reset