VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling

10/07/2020
by   Machel Reid, et al.
0

In this paper, we tackle the task of definition modeling, where the goal is to learn to generate definitions of words and phrases. Existing approaches for this task are discriminative, combining distributional and lexical semantics in an implicit rather than direct way. To tackle this issue we propose a generative model for the task, introducing a continuous latent variable to explicitly model the underlying relationship between a phrase used within a context and its definition. We rely on variational inference for estimation and leverage contextualized word embeddings for improved performance. Our approach is evaluated on four existing challenging benchmarks with the addition of two new datasets, "Cambridge" and the first non-English corpus "Robert", which we release to complement our empirical study. Our Variational Contextual Definition Modeler (VCDM) achieves state-of-the-art performance in terms of automatic and human evaluation metrics, demonstrating the effectiveness of our approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/01/2016

Definition Modeling: Learning to define word embeddings in natural language

Distributed representations of words have been shown to capture lexical ...
research
06/14/2023

"Definition Modeling: To model definitions." Generating Definitions With Little to No Semantics

Definition Modeling, the task of generating definitions, was first propo...
research
12/12/2019

Improving Interpretability of Word Embeddings by Generating Definition and Usage

Word Embeddings, which encode semantic and syntactic features, have achi...
research
11/13/2019

Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling

Defining words in a textual context is a useful task both for practical ...
research
06/26/2018

Conditional Generators of Words Definitions

We explore recently introduced definition modeling technique that provid...
research
05/29/2023

Vec2Gloss: definition modeling leveraging contextualized vectors with Wordnet gloss

Contextualized embeddings are proven to be powerful tools in multiple NL...
research
04/13/2017

A Neural Model for User Geolocation and Lexical Dialectology

We propose a simple yet effective text- based user geolocation model bas...

Please sign up or login with your details

Forgot password? Click here to reset