DeepAI AI Chat
Log In Sign Up

Improving the Expressiveness of Deep Learning Frameworks with Recursion

by   Eunji Jeong, et al.
Seoul National University

Recursive neural networks have widely been used by researchers to handle applications with recursively or hierarchically structured data. However, embedded control flow deep learning frameworks such as TensorFlow, Theano, Caffe2, and MXNet fail to efficiently represent and execute such neural networks, due to lack of support for recursion. In this paper, we add recursion to the programming model of existing frameworks by complementing their design with recursive execution of dataflow graphs as well as additional APIs for recursive definitions. Unlike iterative implementations, which can only understand the topological index of each node in recursive data structures, our recursive implementation is able to exploit the recursive relationships between nodes for efficient execution based on parallel computation. We present an implementation on TensorFlow and evaluation results with various recursive neural network models, showing that our recursive implementation not only conveys the recursive nature of recursive neural networks better than other implementations, but also uses given resources more effectively to reduce training and inference time.


Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs

For a deep learning model, efficient execution of its computation graph ...

Discovery of Governing Equations with Recursive Deep Neural Networks

Model discovery based on existing data has been one of the major focuses...

Symbolic Techniques for Deep Learning: Challenges and Opportunities

As the number of deep learning frameworks increase and certain ones gain...

How does the Mind store Information?

How we store information in our mind has been a major intriguing open qu...

Recursive Neural Networks Can Learn Logical Semantics

Tree-structured recursive neural networks (TreeRNNs) for sentence meanin...

Fabrik: An Online Collaborative Neural Network Editor

We present Fabrik, an online neural network editor that provides tools t...

Feature Weight Tuning for Recursive Neural Networks

This paper addresses how a recursive neural network model can automatica...