A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distance

12/09/2020
by   Minhui Huang, et al.
0

The Wasserstein distance has become increasingly important in machine learning and deep learning. Despite its popularity, the Wasserstein distance is hard to approximate because of the curse of dimensionality. A recently proposed approach to alleviate the curse of dimensionality is to project the sampled data from the high dimensional probability distribution onto a lower-dimensional subspace, and then compute the Wasserstein distance between the projected data. However, this approach requires to solve a max-min problem over the Stiefel manifold, which is very challenging in practice. The only existing work that solves this problem directly is the RGAS (Riemannian Gradient Ascent with Sinkhorn Iteration) algorithm, which requires to solve an entropy-regularized optimal transport problem in each iteration, and thus can be costly for large-scale problems. In this paper, we propose a Riemannian block coordinate descent (RBCD) method to solve this problem, which is based on a novel reformulation of the regularized max-min problem over the Stiefel manifold. We show that the complexity of arithmetic operations for RBCD to obtain an ϵ-stationary point is O(ϵ^-3). This significantly improves the corresponding complexity of RGAS, which is O(ϵ^-12). Moreover, our RBCD has very low per-iteration complexity, and hence is suitable for large-scale problems. Numerical results on both synthetic and real datasets demonstrate that our method is more efficient than existing methods, especially when the number of sampled data is very large.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/05/2021

Projection Robust Wasserstein Barycenter

Collecting and aggregating information from several probability measures...
06/12/2020

Projection Robust Wasserstein Distance and Riemannian Optimization

Projection robust Wasserstein (PRW) distance, or Wasserstein projection ...
02/12/2018

A Fast Proximal Point Method for Wasserstein Distance

Wasserstein distance plays increasingly important roles in machine learn...
06/04/2022

First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces

From optimal transport to robust dimensionality reduction, a plethora of...
11/15/2019

Coupling Matrix Manifolds and Their Applications in Optimal Transport

Optimal transport (OT) is a powerful tool for measuring the distance bet...
06/22/2022

Robust Bayesian Recourse

Algorithmic recourse aims to recommend an informative feedback to overtu...
10/24/2020

Fast Epigraphical Projection-based Incremental Algorithms for Wasserstein Distributionally Robust Support Vector Machine

Wasserstein Distributionally Robust Optimization (DRO) is concerned with...