Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression

03/15/2018
by   Dong Wang, et al.
0

While the research on convolutional neural networks (CNNs) is progressing quickly, the real-world deployment of these models is often limited by computing resources and memory constraints. In this paper, we address this issue by proposing a novel filter pruning method to compress and accelerate CNNs. Our work is based on the linear relationship identified in different feature map subspaces via visualization of feature maps. Such linear relationship implies that the information in CNNs is redundant. Our method eliminates the redundancy in convolutional filters by applying subspace clustering to feature maps. In this way, most of the representative information in the network can be retained in each cluster. Therefore, our method provides an effective solution to filter pruning for which most existing methods directly remove filters based on simple heuristics. The proposed method is independent of the network structure, thus it can be adopted by any off-the-shelf deep learning libraries. Experiments on different networks and tasks show that our method outperforms existing techniques before fine-tuning, and achieves the state-of-the-art results after fine-tuning.

READ FULL TEXT

page 3

page 6

research
03/29/2022

A Passive Similarity based CNN Filter Pruning for Efficient Acoustic Scene Classification

We present a method to develop low-complexity convolutional neural netwo...
research
12/10/2018

Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression

To compress deep convolutional neural networks (CNNs) with large memory ...
research
09/08/2020

CNNPruner: Pruning Convolutional Neural Networks with Visual Analytics

Convolutional neural networks (CNNs) have demonstrated extraordinarily g...
research
11/10/2018

Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks

Modern Convolutional Neural Networks (CNNs) are complex, encompassing mi...
research
08/15/2016

Every Filter Extracts A Specific Texture In Convolutional Neural Networks

Many works have concentrated on visualizing and understanding the inner ...
research
05/21/2020

Feature Statistics Guided Efficient Filter Pruning

Building compact convolutional neural networks (CNNs) with reliable perf...
research
12/11/2018

Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression

Compressing convolutional neural networks (CNNs) has received ever-incre...

Please sign up or login with your details

Forgot password? Click here to reset