Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

by   Samantha Sherman, et al.

We consider the problem of decomposing higher-order moment tensors, i.e., the sum of symmetric outer products of data vectors. Such a decomposition can be used to estimate the means in a Gaussian mixture model and for other applications in machine learning. The dth-order empirical moment tensor of a set of p observations of n variables is a symmetric d-way tensor. Our goal is to find a low-rank tensor approximation comprising r ≪ p symmetric outer products. The challenge is that forming the empirical moment tensors costs O(pn^d) operations and O(n^d) storage, which may be prohibitively expensive; additionally, the algorithm to compute the low-rank approximation costs O(n^d) per iteration. Our contribution is avoiding formation of the moment tensor, computing the low-rank tensor approximation of the moment tensor implicitly using O(pnr) operations per iteration and no extra memory. This advance opens the door to more applications of higher-order moments since they can now be efficiently computed. We present numerical evidence of the computational savings and show an example of estimating the means for higher-order moments.



There are no comments yet.


page 1

page 2

page 3

page 4


The low-rank approximation of fourth-order partial-symmetric and conjugate partial-symmetric tensor

We present an orthogonal matrix outer product decomposition for the four...

Tensor decomposition for learning Gaussian mixtures from moments

In data processing and machine learning, an important challenge is to re...

A Higher Order Unscented Transform

We develop a new approach for estimating the expected values of nonlinea...

Polar decomposition based algorithms on the product of Stiefel manifolds with applications in tensor approximation

In this paper, based on the matrix polar decomposition, we propose a gen...

Streaming Coresets for Symmetric Tensor Factorization

Factorizing tensors has recently become an important optimization module...

Overcomplete order-3 tensor decomposition, blind deconvolution and Gaussian mixture models

We expand the family of tensors that can be efficiently decomposed and a...

Weighted Tensor Decomposition for Learning Latent Variables with Partial Data

Tensor decomposition methods are popular tools for learning latent varia...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.