Log In Sign Up

GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction

by   Wasi Uddin Ahmad, et al.

Prevalent approaches in cross-lingual relation and event extraction use graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic representations such that models trained on one language can be applied to other languages. However, GCNs lack in modeling long-range dependencies or disconnected words in the dependency tree. To address this challenge, we propose to utilize the self-attention mechanism where we explicitly fuse structural information to learn the dependencies between words at different syntactic distances. We introduce GATE, a Graph Attention Transformer Encoder, and test its cross-lingual transferability on relation and event extraction tasks. We perform rigorous experiments on the widely used ACE05 dataset that includes three typologically different languages: English, Chinese, and Arabic. The evaluation results show that GATE outperforms three recently proposed methods by a large margin. Our detailed analysis reveals that due to the reliance on syntactic dependencies, GATE produces robust representations that facilitate transfer across languages.


page 1

page 2

page 3

page 4


Cross-Lingual Adaptation Using Universal Dependencies

We describe a cross-lingual adaptation method based on syntactic parse t...

Cross-Lingual Relation Extraction with Transformers

Relation extraction (RE) is one of the most important tasks in informati...

Robust Cross-lingual Hypernymy Detection using Dependency Context

Cross-lingual Hypernymy Detection involves determining if a word in one ...

Cross-Lingual Relevance Transfer for Document Retrieval

Recent work has shown the surprising ability of multi-lingual BERT to se...

Near or Far, Wide Range Zero-Shot Cross-Lingual Dependency Parsing

Cross-lingual transfer is the major means toleverage knowledge from high...

GTN-ED: Event Detection Using Graph Transformer Networks

Recent works show that the graph structure of sentences, generated from ...

Joint Event Extraction along Shortest Dependency Paths using Graph Convolutional Networks

Event extraction (EE) is one of the core information extraction tasks, w...