DeepAI AI Chat
Log In Sign Up

Embedding Lexical Features via Low-Rank Tensors

by   Mo Yu, et al.
Johns Hopkins University
Carnegie Mellon University

Modern NLP models rely heavily on engineered features, which often combine word and contextual information into complex lexical features. Such combination results in large numbers of features, which can lead to over-fitting. We present a new model that represents complex lexical features---comprised of parts for words, contextual information and labels---in a tensor that captures conjunction information among these parts. We apply low-rank tensor approximations to the corresponding parameter tensors to reduce the parameter space and improve prediction speed. Furthermore, we investigate two methods for handling features that include n-grams of mixed lengths. Our model achieves state-of-the-art results on tasks in relation extraction, PP-attachment, and preposition disambiguation.


page 1

page 2

page 3

page 4


Tensor Low-Rank Reconstruction for Semantic Segmentation

Context information plays an indispensable role in the success of semant...

LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution

Lexical substitution is the task of generating meaningful substitutes fo...

Analyzing Large and Sparse Tensor Data using Spectral Low-Rank Approximation

Information is extracted from large and sparse data sets organized as 3-...

Molding CNNs for text: non-linear, non-consecutive convolutions

The success of deep learning often derives from well-chosen operational ...

Antonym-Synonym Classification Based on New Sub-space Embeddings

Distinguishing antonyms from synonyms is a key challenge for many NLP ap...

NYTWIT: A Dataset of Novel Words in the New York Times

We present the New York Times Word Innovation Types dataset, or NYTWIT, ...