The Expressive Power of Word Embeddings

01/15/2013
by   Yanqing Chen, et al.
0

We seek to better understand the difference in quality of the several publicly released embeddings. We propose several tasks that help to distinguish the characteristics of different embeddings. Our evaluation of sentiment polarity and synonym/antonym relations shows that embeddings are able to capture surprisingly nuanced semantics even in the absence of sentence structure. Moreover, benchmarking the embeddings shows great variance in quality and characteristics of the semantics captured by the tested embeddings. Finally, we show the impact of varying the number of dimensions and the resolution of each dimension on the effective useful features captured by the embedding space. Our contributions highlight the importance of embeddings for NLP tasks and the effect of their quality on the final results.

READ FULL TEXT

page 5

page 8

research
06/02/2023

Word Embeddings for Banking Industry

Applications of Natural Language Processing (NLP) are plentiful, from se...
research
08/18/2015

Learning Meta-Embeddings by Using Ensembles of Embedding Sets

Word embeddings -- distributed representations of words -- in deep learn...
research
02/24/2016

Ultradense Word Embeddings by Orthogonal Transformation

Embeddings are generic representations that are useful for many NLP task...
research
10/05/2019

On Dimensional Linguistic Properties of the Word Embedding Space

Word embeddings have become a staple of several natural language process...
research
06/04/2019

Pitfalls in the Evaluation of Sentence Embeddings

Deep learning models continuously break new records across different NLP...
research
03/12/2018

Concept2vec: Metrics for Evaluating Quality of Embeddings for Ontological Concepts

Although there is an emerging trend towards generating embeddings for pr...
research
11/24/2017

An Exploration of Word Embedding Initialization in Deep-Learning Tasks

Word embeddings are the interface between the world of discrete units of...

Please sign up or login with your details

Forgot password? Click here to reset