Tensor Moments of Gaussian Mixture Models: Theory and Applications

02/14/2022
by   João M. Pereira, et al.
46

Gaussian mixture models (GMM) are fundamental tools in statistical and data sciences. We study the moments of multivariate Gaussians and GMMs. The d-th moment of an n-dimensional random variable is a symmetric d-way tensor of size n^d, so working with moments naively is assumed to be prohibitively expensive for d>2 and larger values of n. In this work, we develop theory and numerical methods for implicit computations with moment tensors of GMMs, reducing the computational and storage costs to 𝒪(n^2) and 𝒪(n^3), respectively, for general covariance matrices, and to 𝒪(n) and 𝒪(n), respectively, for diagonal ones. We derive concise analytic expressions for the moments in terms of symmetrized tensor products, relying on the correspondence between symmetric tensors and homogeneous polynomials, and combinatorial identities involving Bell polynomials. The primary application of this theory is to estimating GMM parameters from a set of observations, when formulated as a moment-matching optimization problem. If there is a known and common covariance matrix, we also show it is possible to debias the data observations, in which case the problem of estimating the unknown means reduces to symmetric CP tensor decomposition. Numerical results validate and illustrate the numerical efficiency of our approaches. This work potentially opens the door to the competitiveness of the method of moments as compared to expectation maximization methods for parameter estimation of GMMs.

READ FULL TEXT

page 9

page 13

page 14

page 16

page 20

page 24

research
11/10/2019

Estimating Higher-Order Moments Using Symmetric Tensor Decomposition

We consider the problem of decomposing higher-order moment tensors, i.e....
research
02/08/2021

Learning Diagonal Gaussian Mixture Models and Incomplete Tensor Decompositions

This paper studies how to learn parameters in diagonal Gaussian mixture ...
research
10/25/2022

Moment Estimation for Nonparametric Mixture Models Through Implicit Tensor Decomposition

We present an alternating least squares type numerical optimization sche...
research
07/16/2020

Overcomplete order-3 tensor decomposition, blind deconvolution and Gaussian mixture models

We expand the family of tensors that can be efficiently decomposed and a...
research
04/15/2021

Polynomial methods in statistical inference: theory and practice

This survey provides an exposition of a suite of techniques based on the...
research
07/19/2018

Optimal estimation of Gaussian mixtures via denoised method of moments

The Method of Moments [Pea94] is one of the most widely used methods in ...
research
01/22/2023

Moment Varieties for Mixtures of Products

The setting of this article is nonparametric algebraic statistics. We st...

Please sign up or login with your details

Forgot password? Click here to reset