Minibatch optimal transport distances; analysis and applications

01/05/2021
by   Kilian Fatras, et al.
9

Optimal transport distances have become a classic tool to compare probability distributions and have found many applications in machine learning. Yet, despite recent algorithmic developments, their complexity prevents their direct use on large scale datasets. To overcome this challenge, a common workaround is to compute these distances on minibatches i.e. to average the outcome of several smaller optimal transport problems. We propose in this paper an extended analysis of this practice, which effects were previously studied in restricted cases. We first consider a large variety of Optimal Transport kernels. We notably argue that the minibatch strategy comes with appealing properties such as unbiased estimators, gradients and a concentration bound around the expectation, but also with limits: the minibatch OT is not a distance. To recover some of the lost distance axioms, we introduce a debiased minibatch OT function and study its statistical and optimisation properties. Along with this theoretical analysis, we also conduct empirical experiments on gradient flows, generative adversarial networks (GANs) or color transfer that highlight the practical interest of this strategy.

READ FULL TEXT

page 21

page 29

page 30

page 31

page 33

page 34

research
10/09/2019

Learning with minibatch Wasserstein : asymptotic and gradient properties

Optimal transport distances are powerful tools to compare probability di...
research
03/05/2021

Unbalanced minibatch Optimal Transport; applications to Domain Adaptation

Optimal transport distances have found many applications in machine lear...
research
12/28/2020

Comparing Probability Distributions with Conditional Transport

To measure the difference between two probability distributions, we prop...
research
06/05/2018

Pathwise Derivatives Beyond the Reparameterization Trick

We observe that gradients computed via the reparameterization trick are ...
research
11/30/2022

Generative Adversarial Learning of Sinkhorn Algorithm Initializations

The Sinkhorn algorithm (arXiv:1306.0895) is the state-of-the-art to comp...
research
02/25/2021

Improving Approximate Optimal Transport Distances using Quantization

Optimal transport (OT) is a popular tool in machine learning to compare ...
research
10/22/2020

MinMax Methods for Optimal Transport and Beyond: Regularization, Approximation and Numerics

We study MinMax solution methods for a general class of optimization pro...

Please sign up or login with your details

Forgot password? Click here to reset