Full-Stack Filters to Build Minimum Viable CNNs

08/06/2019
by   Kai Han, et al.
2

Deep convolutional neural networks (CNNs) are usually over-parameterized, which cannot be easily deployed on edge devices such as mobile phones and smart cameras. Existing works used to decrease the number or size of requested convolution filters for a minimum viable CNN on edge devices. In contrast, this paper introduces filters that are full-stack and can be used to generate many more sub-filters. Weights of these sub-filters are inherited from full-stack filters with the help of different binary masks. Orthogonal constraints are applied over binary masks to decrease their correlation and promote the diversity of generated sub-filters. To preserve the same volume of output feature maps, we can naturally reduce the number of established filters by only maintaining a few full-stack filters and a set of binary masks. We also conduct theoretical analysis on the memory cost and an efficient implementation is introduced for the convolution of the proposed filters. Experiments on several benchmark datasets and CNN models demonstrate that the proposed method is able to construct minimum viable convolution networks of comparable performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/06/2020

Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filters

Deep Convolutional Neural Networks (CNN) have been successfully applied ...
research
09/20/2021

Learning Versatile Convolution Filters for Efficient Visual Recognition

This paper introduces versatile filters to construct efficient convoluti...
research
07/25/2017

Towards Evolutional Compression

Compressing convolutional neural networks (CNNs) is essential for transf...
research
08/28/2018

Convolutional Neural Networks with Recurrent Neural Filters

We introduce a class of convolutional neural networks (CNNs) that utiliz...
research
02/27/2019

Modulated binary cliquenet

Although Convolutional Neural Networks (CNNs) achieve effectiveness in v...
research
01/04/2022

Binary Fuse Filters: Fast and Smaller Than Xor Filters

Bloom and cuckoo filters provide fast approximate set membership while u...
research
09/25/2020

Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters

Convolution is the main building block of convolutional neural networks ...

Please sign up or login with your details

Forgot password? Click here to reset