DeepAI
Log In Sign Up

DyNet: The Dynamic Neural Network Toolkit

01/15/2017
by   Graham Neubig, et al.
0

We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives. In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input. Dynamic declaration thus facilitates the implementation of more complicated network architectures, and DyNet is specifically designed to allow users to implement their models in a way that is idiomatic in their preferred programming language (C++ or Python). One challenge with dynamic declaration is that because the symbolic computation graph is defined anew for every training example, its construction must have low overhead. To achieve this, DyNet has an optimized C++ backend and lightweight graph representation. Experiments show that DyNet's speeds are faster than or comparable with static declaration toolkits, and significantly faster than Chainer, another dynamic declaration toolkit. DyNet is released open-source under the Apache 2.0 license and available at http://github.com/clab/dynet.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/11/2017

Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks

Recent deep learning (DL) models have moved beyond static network archit...
05/22/2017

On-the-fly Operation Batching in Dynamic Computation Graphs

Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offe...
10/05/2016

LAYERS: Yet another Neural Network toolkit

Layers is an open source neural network toolkit aim at providing an easy...
09/17/2021

Natlog: a Lightweight Logic Programming Language with a Neuro-symbolic Touch

We introduce Natlog, a lightweight Logic Programming language, sharing P...
10/21/2020

NeuSpell: A Neural Spelling Correction Toolkit

We introduce NeuSpell, an open-source toolkit for spelling correction in...
02/09/2020

MOGPTK: The Multi-Output Gaussian Process Toolkit

We present MOGPTK, a Python package for multi-channel data modelling usi...
12/04/2020

What the new RooFit can do for your analysis

RooFit is a toolkit for statistical modelling and fitting, and together ...

Code Repositories

dynet

DyNet: The Dynamic Neural Network Toolkit


view repo