Neural Speed Reading via Skim-RNN

11/06/2017
by   Minjoon Seo, et al.
0

Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be easily used instead of RNNs in existing models. In our experiments, we show that Skim-RNN can achieve significantly reduced computational cost without losing accuracy compared to standard RNNs across five different natural language tasks. In addition, we demonstrate that the trade-off between accuracy and speed of Skim-RNN can be dynamically controlled during inference time in a stable manner. Our analysis also shows that Skim-RNN running on a single CPU offers lower latency compared to standard RNNs on GPUs.

READ FULL TEXT
research
11/18/2017

MinimalRNN: Toward More Interpretable and Trainable Recurrent Neural Networks

We introduce MinimalRNN, a new recurrent neural network architecture tha...
research
02/23/2022

NeuroView-RNN: It's About Time

Recurrent Neural Networks (RNNs) are important tools for processing sequ...
research
10/02/2019

AntMan: Sparse Low-Rank Compression to Accelerate RNN inference

Wide adoption of complex RNN based models is hindered by their inference...
research
11/28/2016

Input Switched Affine Networks: An RNN Architecture Designed for Interpretability

There exist many problem domains where the interpretability of neural ne...
research
08/28/2023

Kernel Limit of Recurrent Neural Networks Trained on Ergodic Data Sequences

Mathematical methods are developed to characterize the asymptotics of re...
research
08/30/2019

A single-layer RNN can approximate stacked and bidirectional RNNs, and topologies in between

To enhance the expressiveness and representational capacity of recurrent...
research
12/09/2022

Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement

Can we take a recurrent neural network (RNN) trained to translate betwee...

Please sign up or login with your details

Forgot password? Click here to reset