Low-Resource Parsing with Crosslingual Contextualized Representations

09/19/2019
by   Phoebe Mulcaire, et al.
0

Despite advances in dependency parsing, languages with small treebanks still present challenges. We assess recent approaches to multilingual contextual word representations (CWRs), and compare them for crosslingual transfer from a language with a large treebank to a language with a small or nonexistent treebank, by sharing parameters between languages in the parser itself. We experiment with a diverse selection of languages in both simulated and truly low-resource scenarios, and show that multilingual CWRs greatly facilitate low-resource dependency parsing even without crosslingual supervision such as dictionaries or parallel text. Furthermore, we examine the non-contextual part of the learned language models (which we call a "decontextual probe") to demonstrate that polyglot language models better encode crosslingual lexical correspondence compared to aligned monolingual language models. This analysis provides further evidence that polyglot training is an effective approach to crosslingual transfer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2021

Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?

Recent impressive improvements in NLP, largely based on the success of c...
research
02/26/2019

Polyglot Contextual Representations Improve Crosslingual Transfer

We introduce a method to produce multilingual contextual word representa...
research
06/08/2021

A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021

We evaluate three leading dependency parser systems from different parad...
research
09/29/2020

Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank

Pretrained multilingual contextual representations have shown great succ...
research
04/29/2020

UDapter: Language Adaptation for Truly Universal Dependency Parsing

Recent advances in the field of multilingual dependency parsing have bro...
research
11/11/2019

Deep Contextualized Self-training for Low Resource Dependency Parsing

Neural dependency parsing has proven very effective, achieving state-of-...
research
05/03/2023

evaluating bert and parsbert for analyzing persian advertisement data

This paper discusses the impact of the Internet on modern trading and th...

Please sign up or login with your details

Forgot password? Click here to reset