Spectral dimensionality reduction for HMMs

03/28/2012
by   Dean P. Foster, et al.
0

Hidden Markov Models (HMMs) can be accurately approximated using co-occurrence frequencies of pairs and triples of observations by using a fast spectral method in contrast to the usual slow methods like EM or Gibbs sampling. We provide a new spectral method which significantly reduces the number of model parameters that need to be estimated, and generates a sample complexity that does not depend on the size of the observation vocabulary. We present an elementary proof giving bounds on the relative accuracy of probability estimates from our model. (Correlaries show our bounds can be weakened to provide either L1 bounds or KL bounds which provide easier direct comparisons to previous work.) Our theorem uses conditions that are checkable from the data, instead of putting conditions on the unobservable Markov transition matrix.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2015

Spectral Learning of Large Structured HMMs for Comparative Epigenomics

We develop a latent variable model and an efficient spectral algorithm m...
research
06/24/2021

Fundamental limits for learning hidden Markov model parameters

We study the frontier between learnable and unlearnable hidden Markov mo...
research
12/10/2013

Guaranteed Model Order Estimation and Sample Complexity Bounds for LDA

The question of how to determine the number of independent latent factor...
research
02/19/2018

Learning Hidden Markov Models from Pairwise Co-occurrences with Applications to Topic Modeling

We present a new algorithm for identifying the transition and emission p...
research
02/15/2021

Online learning of Riemannian hidden Markov models in homogeneous Hadamard spaces

Hidden Markov models with observations in a Euclidean space play an impo...
research
10/07/2022

Estimation of the Order of Non-Parametric Hidden Markov Models using the Singular Values of an Integral Operator

We are interested in assessing the order of a finite-state Hidden Markov...
research
12/11/2018

Predictive Learning on Sign-Valued Hidden Markov Trees

We provide high-probability sample complexity guarantees for exact struc...

Please sign up or login with your details

Forgot password? Click here to reset