Parallelizable Stack Long Short-Term Memory

04/06/2019
by   Shuoyang Ding, et al.
0

Stack Long Short-Term Memory (StackLSTM) is useful for various applications such as parsing and string-to-tree neural machine translation, but it is also known to be notoriously difficult to parallelize for GPU training due to the fact that the computations are dependent on discrete operations. In this paper, we tackle this problem by utilizing state access patterns of StackLSTM to homogenize computations with regard to different discrete operations. Our parsing experiments show that the method scales up almost linearly with increasing batch size, and our parallelized PyTorch implementation trains significantly faster compared to the Dynet C++ implementation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/31/2015

Top-down Tree Long Short-Term Memory Networks

Long Short-Term Memory (LSTM) networks, a type of recurrent neural netwo...
research
06/06/2018

Automating Network Error Detection using Long-Short Term Memory Networks

In this work, we investigate the current flaws with identifying network-...
research
02/09/2016

Associative Long Short-Term Memory

We investigate a new method to augment recurrent neural networks with ex...
research
08/06/2020

Respiratory Sound Classification Using Long-Short Term Memory

Developing a reliable sound detection and recognition system offers many...
research
05/29/2015

Transition-Based Dependency Parsing with Stack Long Short-Term Memory

We propose a technique for learning representations of parser states in ...
research
06/29/2016

Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs

We present a transition-based parser that jointly produces syntactic and...
research
02/05/2019

Deep Tree Transductions - A Short Survey

The paper surveys recent extensions of the Long-Short Term Memory networ...

Please sign up or login with your details

Forgot password? Click here to reset