Zero-shot Entity Linking with Efficient Long Range Sequence Modeling

10/12/2020
by   Zonghai Yao, et al.
0

This paper considers the problem of zero-shot entity linking, in which a link in the test time may not present in training. Following the prevailing BERT-based research efforts, we find a simple yet effective way is to expand the long-range sequence modeling. Unlike many previous methods, our method does not require expensive pre-training of BERT with long position embedding. Instead, we propose an efficient position embeddings initialization method called Embedding-repeat, which initializes larger position embeddings based on BERT-Base. On Wikia's zero-shot EL dataset, our method improves the SOTA from 76.06 74.57 sequence modeling without retraining the BERT model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/10/2019

Zero-shot Entity Linking with Dense Entity Retrieval

We consider the zero-shot entity-linking challenge where each entity is ...
research
10/19/2020

Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking

Cross-language entity linking grounds mentions in multiple languages to ...
research
09/08/2021

NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task–Next Sentence Prediction

Using prompts to utilize language models to perform various downstream t...
research
06/18/2019

Zero-Shot Entity Linking by Reading Entity Descriptions

We present the zero-shot entity linking task, where mentions must be lin...
research
03/03/2021

OAG-BERT: Pre-train Heterogeneous Entity-augmented Academic Language Model

To enrich language models with domain knowledge is crucial but difficult...
research
05/23/2023

Prompt position really matters in few-shot and zero-shot NLU tasks

Prompt-based models have made remarkable advancements in the fields of z...
research
05/01/2020

Identifying Necessary Elements for BERT's Multilinguality

It has been shown that multilingual BERT (mBERT) yields high quality mul...

Please sign up or login with your details

Forgot password? Click here to reset