DeepAI AI Chat
Log In Sign Up

Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings

by   Yadollah Yaghoobzadeh, et al.

Word embeddings typically represent different meanings of a word in a single conflated vector. Empirical analysis of embeddings of ambiguous words is currently limited by the small size of manually annotated resources and by the fact that word senses are treated as unrelated individual concepts. We present a large dataset based on manual Wikipedia annotations and word senses, where word senses from different words are related by semantic classes. This is the basis for novel diagnostic tests for an embedding's content: we probe word embeddings for semantic classes and analyze the embedding space by classifying embeddings into semantic classes. Our main findings are: (i) Information about a sense is generally represented well in a single-vector embedding - if the sense is frequent. (ii) A classifier can accurately predict whether a word is single-sense or multi-sense, based only on its embedding. (iii) Although rare senses are not well represented in single-vector embeddings, this does not have negative impact on an NLP application whose performance depends on frequent senses.


page 1

page 2

page 3

page 4


Humor in Word Embeddings: Cockamamie Gobbledegook for Nincompoops

We study humor in Word Embeddings, a popular AI tool that associates eac...

Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications

Most unsupervised NLP models represent each word with a single point or ...

You Shall Know the Most Frequent Sense by the Company it Keeps

Unsupervised identification of the most frequent sense of a polysemous w...

Embedding Words and Senses Together via Joint Knowledge-Enhanced Training

Word embeddings are widely used in Natural Language Processing, mainly d...

Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge

Understanding context-dependent variation in word meanings is a key aspe...

Detecting New Word Meanings: A Comparison of Word Embedding Models in Spanish

Semantic neologisms (SN) are defined as words that acquire a new word me...

A Game-Theoretic Approach to Word Sense Disambiguation

This paper presents a new model for word sense disambiguation formulated...