DeepAI
Log In Sign Up

Rank Selection of CP-decomposed Convolutional Layers with Variational Bayesian Matrix Factorization

01/16/2018
by   Marcella Astrid, et al.
0

Convolutional Neural Networks (CNNs) is one of successful method in many areas such as image classification tasks. However, the amount of memory and computational cost needed for CNNs inference obstructs them to run efficiently in mobile devices because of memory and computational ability limitation. One of the method to compress CNNs is compressing the layers iteratively, i.e. by layer-by-layer compression and fine-tuning, with CP-decomposition in convolutional layers. To compress with CP-decomposition, rank selection is important. In the previous approach rank selection that is based on sensitivity of each layer, the average rank of the network was still arbitrarily selected. Additionally, the rank of all layers were decided before whole process of iterative compression, while the rank of a layer can be changed after fine-tuning. Therefore, this paper proposes selecting rank of each layer using Variational Bayesian Matrix Factorization (VBMF) which is more systematic than arbitrary approach. Furthermore, to consider the change of each layer's rank after fine-tuning of previous iteration, the method is applied just before compressing the target layer, i.e. after fine-tuning of the previous iteration. The results show better accuracy while also having more compression rate in AlexNet's convolutional layers compression.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/19/2014

Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition

We propose a simple two-step approach for speeding up convolution layers...
10/19/2018

CNN inference acceleration using dictionary of centroids

It is well known that multiplication operations in convolutional layers ...
05/21/2021

Compressing Deep CNNs using Basis Representation and Spectral Fine-tuning

We propose an efficient and straightforward method for compressing deep ...
02/06/2020

Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs

In this paper, we propose a fixed convolutional layer with an order of s...
05/28/2020

CPAC-Conv: CP-decomposition to Approximately Compress Convolutional Layers in Deep Learning

Feature extraction for tensor data serves as an important step in many t...
03/12/2019

Cascaded Projection: End-to-End Network Compression and Acceleration

We propose a data-driven approach for deep convolutional neural network ...
09/11/2020

Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation

A common technique for compressing a neural network is to compute the k-...