Learning Embeddings into Entropic Wasserstein Spaces

05/08/2019
by   Charlie Frogner, et al.
10

Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete probability distributions in a Wasserstein space, endowed with an optimal transport metric. Wasserstein spaces are much larger and more flexible than Euclidean spaces, in that they can successfully embed a wider variety of metric structures. We exploit this flexibility by learning an embedding that captures semantic information in the Wasserstein distance between embedded distributions. We examine empirically the representational capacity of our learned Wasserstein embeddings, showing that they can embed a wide variety of metric structures with smaller distortion than an equivalent Euclidean embedding. We also investigate an application to word embedding, demonstrating a unique advantage of Wasserstein embeddings: We can visualize the high-dimensional embedding directly, since it is a probability distribution on a low-dimensional space. This obviates the need for dimensionality reduction techniques like t-SNE for visualization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/19/2018

Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions

Embedding complex objects as vectors in low dimensional spaces is a long...
research
12/04/2019

Deep Distributional Sequence Embeddings Based on a Wasserstein Loss

Deep metric learning employs deep neural networks to embed instances int...
research
10/20/2017

Learning Wasserstein Embeddings

The Wasserstein distance received a lot of attention recently in the com...
research
07/22/2022

Exploring Wasserstein Distance across Concept Embeddings for Ontology Matching

Measuring the distance between ontological elements is a fundamental com...
research
09/25/2019

Optimal Transport to a Variety

We study the problem of minimizing the Wasserstein distance between a pr...
research
02/13/2019

Wasserstein Barycenter Model Ensembling

In this paper we propose to perform model ensembling in a multiclass or ...
research
09/07/2020

Ensemble Riemannian Data Assimilation over the Wasserstein Space

In this paper, we present a new ensemble data assimilation paradigm over...

Please sign up or login with your details

Forgot password? Click here to reset