Multimodal Word Distributions

04/27/2017
by   Ben Athiwaratkun, et al.
0

Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective. We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2019

Learning Multi-Sense Word Distributions using Approximate Kullback-Leibler Divergence

Learning word representations has garnered greater attention in the rece...
research
04/26/2018

Hierarchical Density Order Embeddings

By representing words with probability densities rather than point vecto...
research
09/17/2019

Semantic Relatedness Based Re-ranker for Text Spotting

Applications such as textual entailment, plagiarism detection or documen...
research
12/20/2014

Word Representations via Gaussian Embedding

Current work in lexical distributed representations maps each word to a ...
research
08/21/2018

Gaussian Word Embedding with a Wasserstein Distance Loss

Comparing with word embedding that based on the point representation, di...
research
10/05/2020

On the Effects of Knowledge-Augmented Data in Word Embeddings

This paper investigates techniques for knowledge injection into word emb...
research
05/16/2020

Learning Probabilistic Sentence Representations from Paraphrases

Probabilistic word embeddings have shown effectiveness in capturing noti...

Please sign up or login with your details

Forgot password? Click here to reset