Improving Event Detection using Contextual Word and Sentence Embeddings

07/02/2020
by   Mariano Maisonnave, et al.
0

The task of Event Detection (ED) is a subfield of Information Extraction (IE) that consists in recognizing event mentions in natural language texts. Several applications can take advantage of an ED system, including alert systems, text summarization, question-answering systems, and any system that needs to extract structured information about events from unstructured texts. ED is a complex task, which is hampered by two main challenges: the lack of a dataset large enough to train and test the developed models and the variety of event type definitions that exist in the literature. These problems make generalization hard to achieve, resulting in poor adaptation to different domains and targets. The main contribution of this paper is the design, implementation and evaluation of a recurrent neural network model for ED that combines several features. In particular, the paper makes the following contributions: (1) it uses BERT embeddings to define contextual word and contextual sentence embeddings as attributes, which to the best of our knowledge were never used before for the ED task; (2) the proposed model has the ability to use its first layer to learn good feature representations; (3) a new public dataset with a general definition of event; (4) an extensive empirical evaluation that includes (i) the exploration of different architectures and hyperparameters, (ii) an ablation test to study the impact of each attribute, and (iii) a comparison with a replication of a state-of-the-art model. The results offer several insights into the importance of contextual embeddings and indicate that the proposed approach is effective in the ED task, outperforming the baseline models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/09/2020

Automated Discovery of Mathematical Definitions in Text with Deep Neural Networks

Automatic definition extraction from texts is an important task that has...
research
08/26/2018

Event Detection with Neural Networks: A Rigorous Empirical Evaluation

Detecting events and classifying them into predefined types is an import...
research
04/01/2019

Unsupervised Abbreviation Disambiguation Contextual disambiguation using word embeddings

As abbreviations often have several distinct meanings, disambiguating th...
research
04/17/2019

Contextual Aware Joint Probability Model Towards Question Answering System

In this paper, we address the question answering challenge with the SQuA...
research
11/22/2019

High Quality ELMo Embeddings for Seven Less-Resourced Languages

Recent results show that deep neural networks using contextual embedding...
research
10/04/2018

Italian Event Detection Goes Deep Learning

This paper reports on a set of experiments with different word embedding...
research
11/15/2022

Type Information Utilized Event Detection via Multi-Channel GNNs in Electrical Power Systems

Event detection in power systems aims to identify triggers and event typ...

Please sign up or login with your details

Forgot password? Click here to reset