Tensor Moments of Gaussian Mixture Models: Theory and Applications

by   João M. Pereira, et al.

Gaussian mixture models (GMM) are fundamental tools in statistical and data sciences. We study the moments of multivariate Gaussians and GMMs. The d-th moment of an n-dimensional random variable is a symmetric d-way tensor of size n^d, so working with moments naively is assumed to be prohibitively expensive for d>2 and larger values of n. In this work, we develop theory and numerical methods for implicit computations with moment tensors of GMMs, reducing the computational and storage costs to 𝒪(n^2) and 𝒪(n^3), respectively, for general covariance matrices, and to 𝒪(n) and 𝒪(n), respectively, for diagonal ones. We derive concise analytic expressions for the moments in terms of symmetrized tensor products, relying on the correspondence between symmetric tensors and homogeneous polynomials, and combinatorial identities involving Bell polynomials. The primary application of this theory is to estimating GMM parameters from a set of observations, when formulated as a moment-matching optimization problem. If there is a known and common covariance matrix, we also show it is possible to debias the data observations, in which case the problem of estimating the unknown means reduces to symmetric CP tensor decomposition. Numerical results validate and illustrate the numerical efficiency of our approaches. This work potentially opens the door to the competitiveness of the method of moments as compared to expectation maximization methods for parameter estimation of GMMs.


page 9

page 13

page 14

page 16

page 20

page 24

∙ 11/10/2019

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

We consider the problem of decomposing higher-order moment tensors, i.e....
∙ 02/08/2021

Learning Diagonal Gaussian Mixture Models and Incomplete Tensor Decompositions

This paper studies how to learn parameters in diagonal Gaussian mixture ...
∙ 04/15/2021

Polynomial methods in statistical inference: theory and practice

This survey provides an exposition of a suite of techniques based on the...
∙ 07/16/2020

Overcomplete order-3 tensor decomposition, blind deconvolution and Gaussian mixture models

We expand the family of tensors that can be efficiently decomposed and a...
∙ 06/01/2021

Tensor decomposition for learning Gaussian mixtures from moments

In data processing and machine learning, an important challenge is to re...
∙ 07/19/2018

Optimal estimation of Gaussian mixtures via denoised method of moments

The Method of Moments [Pea94] is one of the most widely used methods in ...
∙ 05/03/2021

A Bayesian Method for Estimating Uncertainty in Excavated Material

This paper proposes a method to probabilistically quantify the moments (...