Geometry of Polysemy

10/24/2016
by   Jiaqi Mu, et al.
0

Vector representations of words have heralded a transformational approach to classical problems in NLP; the most popular example is word2vec. However, a single vector does not suffice to model the polysemous nature of many (frequent) words, i.e., words with multiple meanings. In this paper, we propose a three-fold approach for unsupervised polysemy modeling: (a) context representations, (b) sense induction and disambiguation and (c) lexeme (as a word and sense pair) representations. A key feature of our work is the finding that a sentence containing a target word is well represented by a low rank subspace, instead of a point in a vector space. We then show that the subspaces associated with a particular sense of the target word tend to intersect over a line (one-dimensional subspace), which we use to disambiguate senses using a clustering algorithm that harnesses the Grassmannian geometry of the representations. The disambiguation algorithm, which we call K-Grassmeans, leads to a procedure to label the different senses of the target word in the corpus -- yielding lexeme vector representations, all in an unsupervised manner starting from a large (Wikipedia) corpus in English. Apart from several prototypical target (word,sense) examples and a host of empirical studies to intuit and justify the various geometric representations, we validate our algorithms on standard sense induction and disambiguation datasets and present new state-of-the-art results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/14/2021

Large Scale Substitution-based Word Sense Induction

We present a word-sense induction method based on pre-trained masked lan...
research
01/20/2016

Semantic Word Clusters Using Signed Normalized Graph Cuts

Vector space representations of words capture many aspects of word simil...
research
08/05/2016

De-Conflated Semantic Representations

One major deficiency of most semantic representation techniques is that ...
research
05/28/2019

Automatic Ambiguity Detection

Most work on sense disambiguation presumes that one knows beforehand -- ...
research
07/07/2000

Boosting Applied to Word Sense Disambiguation

In this paper Schapire and Singer's AdaBoost.MH boosting algorithm is ap...
research
12/02/2016

Alleviating Overfitting for Polysemous Words for Word Representation Estimation Using Lexicons

Though there are some works on improving distributed word representation...
research
11/22/2018

AutoSense Model for Word Sense Induction

Word sense induction (WSI), or the task of automatically discovering mul...

Please sign up or login with your details

Forgot password? Click here to reset