On the Compression of Recurrent Neural Networks with an Application to LVCSR acoustic modeling for Embedded Speech Recognition

03/25/2016
by   Rohit Prabhavalkar, et al.
0

We study the problem of compressing recurrent neural networks (RNNs). In particular, we focus on the compression of RNN acoustic models, which are motivated by the goal of building compact and accurate speech recognition systems which can be run efficiently on mobile devices. In this work, we present a technique for general recurrent model compression that jointly compresses both recurrent and non-recurrent inter-layer weight matrices. We find that the proposed technique allows us to reduce the size of our Long Short-Term Memory (LSTM) acoustic model to a third of its original size with negligible loss in accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/24/2015

Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition

We have recently shown that deep Long Short-Term Memory (LSTM) recurrent...
research
04/09/2016

Learning Compact Recurrent Neural Networks

Recurrent neural networks (RNNs), including long short-term memory (LSTM...
research
12/02/2016

Parameter Compression of Recurrent Neural Networks and Degradation of Short-term Memory

The significant computational costs of deploying neural networks in larg...
research
06/04/2018

Dynamically Hierarchy Revolution: DirNet for Compressing Recurrent Neural Network on Mobile Devices

Recurrent neural networks (RNNs) achieve cutting-edge performance on a v...
research
06/05/2019

Complex Evolution Recurrent Neural Networks (ceRNNs)

Unitary Evolution Recurrent Neural Networks (uRNNs) have three attractiv...
research
06/20/2016

Fast, Compact, and High Quality LSTM-RNN Based Statistical Parametric Speech Synthesizers for Mobile Devices

Acoustic models based on long short-term memory recurrent neural network...
research
11/20/2017

E-PUR: An Energy-Efficient Processing Unit for Recurrent Neural Networks

Recurrent Neural Networks (RNNs) are a key technology for emerging appli...

Please sign up or login with your details

Forgot password? Click here to reset