Self-Attention Amortized Distributional Projection Optimization for Sliced Wasserstein Point-Cloud Reconstruction

01/12/2023
by   Khai Nguyen, et al.
5

Max sliced Wasserstein (Max-SW) distance has been widely known as a solution for redundant projections of sliced Wasserstein (SW) distance. In applications that have various independent pairs of probability measures, amortized projection optimization is utilized to predict the “max" projecting directions given two input measures instead of using projected gradient ascent multiple times. Despite being efficient, the first issue of the current framework is the violation of permutation invariance property and symmetry property. To address the issue, we propose to design amortized models based on self-attention architecture. Moreover, we adopt efficient self-attention architectures to make the computation linear in the number of supports. Secondly, Max-SW and its amortized version cannot guarantee metricity property due to the sub-optimality of the projected gradient ascent and the amortization gap. Therefore, we propose to replace Max-SW with distributional sliced Wasserstein distance with von Mises-Fisher (vMF) projecting distribution (v-DSW). Since v-DSW is a metric with any non-degenerate vMF distribution, its amortized version can guarantee the metricity when predicting the best discriminate projecting distribution. With the two improvements, we derive self-attention amortized distributional projection optimization and show its appealing performance in point-cloud reconstruction and its downstream applications.

READ FULL TEXT

page 12

page 23

research
02/18/2020

Distributional Sliced-Wasserstein and Applications to Generative Modeling

Sliced-Wasserstein distance (SWD) and its variation, Max Sliced-Wasserst...
research
04/26/2023

Energy-Based Sliced Wasserstein Distance

The sliced Wasserstein (SW) distance has been widely recognized as a sta...
research
01/10/2023

Markovian Sliced Wasserstein Distances: Beyond Independent Projections

Sliced Wasserstein (SW) distance suffers from redundant projections due ...
research
03/25/2022

Amortized Projection Optimization for Sliced Wasserstein Generative Models

Seeking informative projecting directions has been an important task in ...
research
02/05/2021

Projection Robust Wasserstein Barycenter

Collecting and aggregating information from several probability measures...
research
01/22/2021

Projected Statistical Methods for Distributional Data on the Real Line with the Wasserstein Metric

We present a novel class of projected methods, to perform statistical an...

Please sign up or login with your details

Forgot password? Click here to reset