DeepAI AI Chat
Log In Sign Up

Intrinsic analysis for dual word embedding space models

12/01/2020
by   Mohit Mayank, et al.
0

Recent word embeddings techniques represent words in a continuous vector space, moving away from the atomic and sparse representations of the past. Each such technique can further create multiple varieties of embeddings based on different settings of hyper-parameters like embedding dimension size, context window size and training method. One additional variety appears when we especially consider the Dual embedding space techniques which generate not one but two-word embeddings as output. This gives rise to an interesting question - "is there one or a combination of the two word embeddings variety, which works better for a specific task?". This paper tries to answer this question by considering all of these variations. Herein, we compare two classical embedding methods belonging to two different methodologies - Word2Vec from window-based and Glove from count-based. For an extensive evaluation after considering all variations, a total of 84 different models were compared against semantic, association and analogy evaluations tasks which are made up of 9 open-source linguistics datasets. The final Word2vec reports showcase the preference of non-default model for 2 out of 3 tasks. In case of Glove, non-default models outperform in all 3 evaluation tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/26/2020

Robust and Consistent Estimation of Word Embedding for Bangla Language by fine-tuning Word2Vec Model

Word embedding or vector representation of word holds syntactical and se...
04/05/2023

PWESuite: Phonetic Word Embeddings and Tasks They Facilitate

Word embeddings that map words into a fixed-dimensional vector space are...
06/16/2021

Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

Several variants of deep neural networks have been successfully employed...
02/15/2020

Supervised Phrase-boundary Embeddings

We propose a new word embedding model, called SPhrase, that incorporates...
09/30/2021

Phonetic Word Embeddings

This work presents a novel methodology for calculating the phonetic simi...
08/14/2018

Embedding Grammars

Classic grammars and regular expressions can be used for a variety of pu...
11/06/2015

Towards a Better Understanding of Predict and Count Models

In a recent paper, Levy and Goldberg pointed out an interesting connecti...