Probing for Labeled Dependency Trees

03/24/2022
by   Max Müller-Eberstein, et al.
0

Probing has become an important tool for analyzing representations in Natural Language Processing (NLP). For graphical NLP tasks such as dependency parsing, linear probes are currently limited to extracting undirected or unlabeled parse trees which do not capture the full task. This work introduces DepProbe, a linear probe which can extract labeled and directed dependency parse trees from embeddings while using fewer parameters and compute than prior methods. Leveraging its full task coverage and lightweight parametrization, we investigate its predictive power for selecting the best transfer language for training a full biaffine attention parser. Across 13 languages, our proposed method identifies the best source treebank 94 competitive baselines and prior work. Finally, we analyze the informativeness of task-specific subspaces in contextual embeddings as well as which benefits a full parser's non-linear parametrization provides.

READ FULL TEXT

page 5

page 8

page 14

page 15

page 16

06/10/2022

Sort by Structure: Language Model Ranking as Dependency Probing

Making an informed choice of pre-trained language model (LM) is critical...
06/16/2015

Parsing Natural Language Sentences by Semi-supervised Methods

We present our work on semi-supervised parsing of natural language sente...
11/10/2019

Rethinking Self-Attention: An Interpretable Self-Attentive Encoder-Decoder Parser

Attention mechanisms have improved the performance of NLP tasks while pr...
01/12/2022

Biaffine Discourse Dependency Parsing

We provide a study of using the biaffine model for neural discourse depe...
10/06/2020

Please Mind the Root: Decoding Arborescences for Dependency Parsing

The connection between dependency trees and spanning trees is exploited ...
04/17/2020

Neural Approaches for Data Driven Dependency Parsing in Sanskrit

Data-driven approaches for dependency parsing have been of great interes...
03/06/2020

Parsing Thai Social Data: A New Challenge for Thai NLP

Dependency parsing (DP) is a task that analyzes text for syntactic struc...