Rotations and Interpretability of Word Embeddings: the Case of the Russian Language

07/14/2017
by   Alexey Zobnin, et al.
0

Consider a continuous word embedding model. Usually, the cosines between word vectors are used as a measure of similarity of words. These cosines do not change under orthogonal transformations of the embedding space. We demonstrate that, using some canonical orthogonal transformations from SVD, it is possible both to increase the meaning of some components and to make the components more stable under re-learning. We study the interpretability of components for publicly available models for the Russian language (RusVectores, fastText, RDT).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2019

Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space

A notable property of word embeddings is that word relationships can exi...
research
03/19/2022

From meaning to perception – exploring the space between word and odor perception embeddings

In this paper we propose the use of the Word2vec algorithm in order to o...
research
07/09/2020

Cultural Cartography with Word Embeddings

Using the presence or frequency of keywords is a classic approach in the...
research
03/08/2018

Improving Optimization in Models With Continuous Symmetry Breaking

Many loss functions in representation learning are invariant under a con...
research
04/05/2017

Linear Ensembles of Word Embedding Models

This paper explores linear methods for combining several word embedding ...
research
05/22/2023

LM-Switch: Lightweight Language Model Conditioning in Word Embedding Space

In recent years, large language models (LMs) have achieved remarkable pr...
research
11/06/2019

Invariance and identifiability issues for word embeddings

Word embeddings are commonly obtained as optimizers of a criterion funct...

Please sign up or login with your details

Forgot password? Click here to reset