Language Model Priming for Cross-Lingual Event Extraction

09/25/2021
by   Steven Fincke, et al.
0

We present a novel, language-agnostic approach to "priming" language models for the task of event extraction, providing particularly effective performance in low-resource and zero-shot cross-lingual settings. With priming, we augment the input to the transformer stack's language model differently depending on the question(s) being asked of the model at runtime. For instance, if the model is being asked to identify arguments for the trigger "protested", we will provide that trigger as part of the input to the language model, allowing it to produce different representations for candidate arguments than when it is asked about arguments for the trigger "arrest" elsewhere in the same sentence. We show that by enabling the language model to better compensate for the deficits of sparse and noisy training data, our approach improves both trigger and argument detection and classification significantly over the state of the art in a zero-shot cross-lingual setting.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/10/2019

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning

Pretrained language models are promising particularly for low-resource l...
10/16/2020

Cross-Lingual Relation Extraction with Transformers

Relation extraction (RE) is one of the most important tasks in informati...
11/28/2021

Zero-Shot Cross-Lingual Transfer in Legal Domain Using Transformer Models

Zero-shot cross-lingual transfer is an important feature in modern NLP m...
03/31/2017

One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

We present a novel cross-lingual transfer method for paradigm completion...
01/26/2021

Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks

In zero-shot cross-lingual transfer, a supervised NLP task trained on a ...
04/11/2020

LAReQA: Language-agnostic answer retrieval from a multilingual pool

We present LAReQA, a challenging new benchmark for language-agnostic ans...
04/28/2020

Event Extraction by Answering (Almost) Natural Questions

The problem of event extraction requires detecting the event trigger and...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.