DeepAI AI Chat
Log In Sign Up

DenseHMM: Learning Hidden Markov Models by Learning Dense Representations

by   Joachim Sicking, et al.

We propose DenseHMM - a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. Our approach enables constraint-free and gradient-based optimization. We propose two optimization schemes that make use of this: a modification of the Baum-Welch algorithm and a direct co-occurrence optimization. The latter one is highly scalable and comes empirically without loss of performance compared to standard HMMs. We show that the non-linearity of the kernelization is crucial for the expressiveness of the representations. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.


A Simple Explanation of A Spectral Algorithm for Learning Hidden Markov Models

A simple linear algebraic explanation of the algorithm in "A Spectral Al...

Learning Hidden Quantum Markov Models

Hidden Quantum Markov Models (HQMMs) can be thought of as quantum probab...

Learning Hidden Markov Models from Pairwise Co-occurrences with Applications to Topic Modeling

We present a new algorithm for identifying the transition and emission p...

An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov models

Non-homogeneous hidden Markov models (NHHMM) are a subclass of dependent...

Clustering-Enhanced Stochastic Gradient MCMC for Hidden Markov Models with Rare States

MCMC algorithms for hidden Markov models, which often rely on the forwar...

Scaling Hidden Markov Language Models

The hidden Markov model (HMM) is a fundamental tool for sequence modelin...

A quantum learning approach based on Hidden Markov Models for failure scenarios generation

Finding the failure scenarios of a system is a very complex problem in t...