Flattened Convolutional Neural Networks for Feedforward Acceleration

12/17/2014
by   Jonghoon Jin, et al.
0

We present flattened convolutional neural networks that are designed for fast feedforward execution. The redundancy of the parameters, especially weights of the convolutional filters in convolutional neural networks has been extensively studied and different heuristics have been proposed to construct a low rank basis of the filters after training. In this work, we train flattened networks that consist of consecutive sequence of one-dimensional filters across all directions in 3D space to obtain comparable performance as conventional convolutional networks. We tested flattened model on different datasets and found that the flattened layer can effectively substitute for the 3D filters without loss of accuracy. The flattened convolution pipelines provide around two times speed-up during feedforward pass compared to the baseline model due to the significant reduction of learning parameters. Furthermore, the proposed method does not require efforts in manual tuning or post processing once the model is trained.

READ FULL TEXT
research
06/15/2018

Bayesian Convolutional Neural Networks

We propose a Bayesian convolutional neural network built upon Bayes by B...
research
08/13/2018

Rank-1 Convolutional Neural Network

In this paper, we propose a convolutional neural network(CNN) with 3-D r...
research
11/20/2015

Training CNNs with Low-Rank Filters for Efficient Image Classification

We propose a new method for creating computationally efficient convoluti...
research
05/15/2014

Speeding up Convolutional Neural Networks with Low Rank Expansions

The focus of this paper is speeding up the evaluation of convolutional n...
research
01/23/2020

DCT-Conv: Coding filters in convolutional networks with Discrete Cosine Transform

Convolutional neural networks are based on a huge number of trained weig...
research
09/30/2015

Fast Algorithms for Convolutional Neural Networks

Deep convolutional neural networks take GPU days of compute time to trai...
research
10/28/2018

Distilling Critical Paths in Convolutional Neural Networks

Neural network compression and acceleration are widely demanded currentl...

Please sign up or login with your details

Forgot password? Click here to reset