LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution

07/11/2021
by   George Michalopoulos, et al.
0

Lexical substitution is the task of generating meaningful substitutes for a word in a given textual context. Contextual word embedding models have achieved state-of-the-art results in the lexical substitution task by relying on contextual information extracted from the replaced word within the sentence. However, such models do not take into account structured knowledge that exists in external lexical databases. We introduce LexSubCon, an end-to-end lexical substitution framework based on contextual embedding models that can identify highly accurate substitute candidates. This is achieved by combining contextual information with knowledge from structured lexical resources. Our approach involves: (i) introducing a novel mix-up embedding strategy in the creation of the input embedding of the target word through linearly interpolating the pair of the target input embedding and the average embedding of its probable synonyms; (ii) considering the similarity of the sentence-definition embeddings of the target word and its proposed candidates; and, (iii) calculating the effect of each substitution in the semantics of the sentence through a fine-tuned sentence similarity model. Our experiments show that LexSubCon outperforms previous state-of-the-art methods on LS07 and CoInCo benchmark datasets that are widely used for lexical substitution tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/20/2020

Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources

In this work, we present an effective method for semantic specialization...
research
06/13/2018

How Predictable is Your State? Leveraging Lexical and Contextual Information for Predicting Legislative Floor Action at the State Level

Modeling U.S. Congressional legislation and roll-call votes has received...
research
04/21/2022

An Attention-Based Model for Predicting Contextual Informativeness and Curriculum Learning Applications

Both humans and machines learn the meaning of unknown words through cont...
research
04/02/2016

Embedding Lexical Features via Low-Rank Tensors

Modern NLP models rely heavily on engineered features, which often combi...
research
06/08/2021

Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality

We release a new benchmark for lexical substitution, the task of finding...
research
04/01/2019

PAWS: Paraphrase Adversaries from Word Scrambling

Existing paraphrase identification datasets lack sentence pairs that hav...
research
10/30/2020

Target Word Masking for Location Metonymy Resolution

Existing metonymy resolution approaches rely on features extracted from ...

Please sign up or login with your details

Forgot password? Click here to reset