Deep Distributional Sequence Embeddings Based on a Wasserstein Loss

12/04/2019
by   Ahmed Abdelwahab, et al.
0

Deep metric learning employs deep neural networks to embed instances into a metric space such that distances between instances of the same class are small and distances between instances from different classes are large. In most existing deep metric learning techniques, the embedding of an instance is given by a feature vector produced by a deep neural network and Euclidean distance or cosine similarity defines distances between these vectors. In this paper, we study deep distributional embeddings of sequences, where the embedding of a sequence is given by the distribution of learned deep features across the sequence. This has the advantage of capturing statistical information about the distribution of patterns within the sequence in the embedding. When embeddings are distributions rather than vectors, measuring distances between embeddings involves comparing their respective distributions. We propose a distance metric based on Wasserstein distances between the distributions and a corresponding loss function for metric learning, which leads to a novel end-to-end trainable embedding model. We empirically observe that distributional embeddings outperform standard vector embeddings and that training with the proposed Wasserstein metric outperforms training with other distance functions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/06/2020

Deep Divergence Learning

Classical linear metric learning methods have recently been extended alo...
research
11/05/2020

Deep Metric Learning with Spherical Embedding

Deep metric learning has attracted much attention in recent years, due t...
research
02/21/2020

An end-to-end approach for the verification problem: learning the right distance

In this contribution, we augment the metric learning setting by introduc...
research
10/07/2022

Learning to embed semantic similarity for joint image-text retrieval

We present a deep learning approach for learning the joint semantic embe...
research
05/08/2019

Learning Embeddings into Entropic Wasserstein Spaces

Euclidean embeddings of data are fundamentally limited in their ability ...
research
09/20/2019

Deep Metric Learning using Similarities from Nonlinear Rank Approximations

In recent years, deep metric learning has achieved promising results in ...
research
03/01/2018

PIP Distance: A Unitary-invariant Metric for Understanding Functionality and Dimensionality of Vector Embeddings

In this paper, we present a theoretical framework for understanding vect...

Please sign up or login with your details

Forgot password? Click here to reset