DeepAI AI Chat
Log In Sign Up

DRAGNN: A Transition-based Framework for Dynamically Connected Neural Networks

03/13/2017
by   Lingpeng Kong, et al.
Google
Carnegie Mellon University
0

In this work, we present a compact, modular framework for constructing novel recurrent neural architectures. Our basic module is a new generic unit, the Transition Based Recurrent Unit (TBRU). In addition to hidden layer activations, TBRUs have discrete state dynamics that allow network connections to be built dynamically as a function of intermediate activations. By connecting multiple TBRUs, we can extend and combine commonly used architectures such as sequence-to-sequence, attention mechanisms, and re-cursive tree-structured models. A TBRU can also serve as both an encoder for downstream tasks and as a decoder for its own task simultaneously, resulting in more accurate multi-task learning. We call our approach Dynamic Recurrent Acyclic Graphical Neural Networks, or DRAGNN. We show that DRAGNN is significantly more accurate and efficient than seq2seq with attention for syntactic dependency parsing and yields more accurate multi-task learning for extractive summarization tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/26/2018

Multi-task Learning over Graph Structures

We present two architectures for multi-task learning with neural sequenc...
11/19/2015

Multi-task Sequence to Sequence Learning

Sequence to sequence learning has recently emerged as a new paradigm in ...
09/24/2019

Recurrent Independent Mechanisms

Learning modular structures which reflect the dynamics of the environmen...
05/28/2021

Efficient and robust multi-task learning in the brain with modular task primitives

In a real-world setting biological agents do not have infinite resources...
11/03/2017

Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning

Multi-task learning (MTL) with neural networks leverages commonalities i...
02/01/2022

Multi-Order Networks for Action Unit Detection

Deep multi-task methods, where several tasks are learned within a single...