SWIFT: Scalable Wasserstein Factorization for Sparse Nonnegative Tensors

10/08/2020
by   Ardavan Afshar, et al.
0

Existing tensor factorization methods assume that the input tensor follows some specific distribution (i.e. Poisson, Bernoulli and Gaussian), and solve the factorization by minimizing some empirical loss functions defined based on the corresponding distribution. However, it suffers from several drawbacks: 1) In reality, the underlying distributions are complicated and unknown, making it infeasible to be approximated by a simple distribution. 2) The correlation across dimensions of the input tensor is not well utilized, leading to sub-optimal performance. Although heuristics were proposed to incorporate such correlation as side information under Gaussian distribution, they can not easily be generalized to other distributions. Thus, a more principled way of utilizing the correlation in tensor factorization models is still an open challenge. Without assuming any explicit distribution, we formulate the tensor factorization as an optimal transport problem with Wasserstein distance, which can handle non-negative inputs. We introduce SWIFT, which minimizes the Wasserstein distance that measures the distance between the input tensor and that of the reconstruction. In particular, we define the N-th order tensor Wasserstein loss for the widely used tensor CP factorization, and derive the optimization algorithm that minimizes it. By leveraging sparsity structure and different equivalent formulations for optimizing computational efficiency, SWIFT is as scalable as other well-known CP algorithms. Using the factor matrices as features, SWIFT achieves up to 9.65 improvement over baselines for downstream prediction tasks. Under the noisy conditions, SWIFT achieves up to 15 best competitors for the prediction tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2021

Non-negative matrix and tensor factorisations with a smoothed Wasserstein loss

Non-negative matrix and tensor factorisations are a classical tool in ma...
research
12/15/2014

Bayesian multi-tensor factorization

We introduce Bayesian multi-tensor factorization, a model that is the fi...
research
08/22/2022

Efficient construction of canonical polyadic approximations of tensor networks

We consider the problem of constructing a canonical polyadic (CP) decomp...
research
07/21/2020

Sparse Nonnegative Tensor Factorization and Completion with Noisy Observations

In this paper, we study the sparse nonnegative tensor factorization and ...
research
08/18/2015

Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data

We present a Bayesian non-negative tensor factorization model for count-...
research
10/12/2022

SeKron: A Decomposition Method Supporting Many Factorization Structures

While convolutional neural networks (CNNs) have become the de facto stan...
research
11/08/2022

Robust Manifold Nonnegative Tucker Factorization for Tensor Data Representation

Nonnegative Tucker Factorization (NTF) minimizes the euclidean distance ...

Please sign up or login with your details

Forgot password? Click here to reset