Tensor decompositions for learning latent variable models

10/29/2012
by   Anima Anandkumar, et al.
0

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin's perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/12/2013

Efficient Orthogonal Tensor Decomposition, with an Application to Latent Variable Model Learning

Decomposing tensors into orthogonal factors is a well-known task in stat...
research
04/16/2020

Spectral Learning on Matrices and Tensors

Spectral methods have been the mainstay in several domains such as machi...
research
12/28/2016

Provable learning of Noisy-or Networks

Many machine learning applications use latent variable models to explain...
research
10/18/2017

Weighted Tensor Decomposition for Learning Latent Variables with Partial Data

Tensor decomposition methods are popular tools for learning latent varia...
research
10/17/2018

Hierarchical Methods of Moments

Spectral methods of moments provide a powerful tool for learning the par...
research
02/05/2015

Provable Sparse Tensor Decomposition

We propose a novel sparse tensor decomposition method, namely Tensor Tru...
research
02/27/2018

Learning Binary Latent Variable Models: A Tensor Eigenpair Approach

Latent variable models with hidden binary units appear in various applic...

Please sign up or login with your details

Forgot password? Click here to reset