A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distance

12/09/2020
by   Minhui Huang, et al.
0

The Wasserstein distance has become increasingly important in machine learning and deep learning. Despite its popularity, the Wasserstein distance is hard to approximate because of the curse of dimensionality. A recently proposed approach to alleviate the curse of dimensionality is to project the sampled data from the high dimensional probability distribution onto a lower-dimensional subspace, and then compute the Wasserstein distance between the projected data. However, this approach requires to solve a max-min problem over the Stiefel manifold, which is very challenging in practice. The only existing work that solves this problem directly is the RGAS (Riemannian Gradient Ascent with Sinkhorn Iteration) algorithm, which requires to solve an entropy-regularized optimal transport problem in each iteration, and thus can be costly for large-scale problems. In this paper, we propose a Riemannian block coordinate descent (RBCD) method to solve this problem, which is based on a novel reformulation of the regularized max-min problem over the Stiefel manifold. We show that the complexity of arithmetic operations for RBCD to obtain an ϵ-stationary point is O(ϵ^-3). This significantly improves the corresponding complexity of RGAS, which is O(ϵ^-12). Moreover, our RBCD has very low per-iteration complexity, and hence is suitable for large-scale problems. Numerical results on both synthetic and real datasets demonstrate that our method is more efficient than existing methods, especially when the number of sampled data is very large.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/05/2021

Projection Robust Wasserstein Barycenter

Collecting and aggregating information from several probability measures...
research
06/12/2020

Projection Robust Wasserstein Distance and Riemannian Optimization

Projection robust Wasserstein (PRW) distance, or Wasserstein projection ...
research
02/12/2018

A Fast Proximal Point Method for Wasserstein Distance

Wasserstein distance plays increasingly important roles in machine learn...
research
05/24/2019

Sliced Gromov-Wasserstein

Recently used in various machine learning contexts, the Gromov-Wasserste...
research
05/03/2023

Low-complexity subspace-descent over symmetric positive definite manifold

This work puts forth low-complexity Riemannian subspace descent algorith...
research
01/27/2019

Approximation of Wasserstein distance with Transshipment

An algorithm for approximating the p-Wasserstein distance between histog...
research
06/09/2021

Large-scale optimal transport map estimation using projection pursuit

This paper studies the estimation of large-scale optimal transport maps ...

Please sign up or login with your details

Forgot password? Click here to reset