What Should/Do/Can LSTMs Learn When Parsing Auxiliary Verb Constructions?

by   Miryam de Lhoneux, et al.

This article is a linguistic investigation of a neural parser. We look at transitivity and agreement information of auxiliary verb constructions (AVCs) in comparison to finite main verbs (FMVs). This comparison is motivated by theoretical work in dependency grammar and in particular the work of Tesnière (1959) where AVCs and FMVs are both instances of a nucleus, the basic unit of syntax. An AVC is a dissociated nucleus, it consists of at least two words, and a FMV is its non-dissociated counterpart, consisting of exactly one word. We suggest that the representation of AVCs and FMVs should capture similar information. We use diagnostic classifiers to probe agreement and transitivity information in vectors learned by a transition-based neural parser in four typologically different languages. We find that the parser learns different information about AVCs and FMVs if only sequential models (BiLSTMs) are used in the architecture but similar information when a recursive layer is used. We find explanations for why this is the case by looking closely at how information is learned in the network and looking at what happens with different dependency representations of AVCs.



There are no comments yet.


page 14


Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs

We present extensions to a continuous-state dependency parsing method th...

Arc-Standard Spinal Parsing with Stack-LSTMs

We present a neural transition-based parser for spinal trees, a dependen...

On Multilingual Training of Neural Dependency Parsers

We show that a recently proposed neural dependency parser can be improve...

Parsing using a grammar of word association vectors

This paper was was first drafted in 2001 as a formalization of the syste...

Easy-First Dependency Parsing with Hierarchical Tree LSTMs

We suggest a compositional vector representation of parse trees that rel...

Stack-propagation: Improved Representation Learning for Syntax

Traditional syntax models typically leverage part-of-speech (POS) inform...

Learning to Search for Dependencies

We demonstrate that a dependency parser can be built using a credit assi...

Code Repositories


Code for a paper about analysing auxiliary verb constructions in parsing

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.