DeepAI AI Chat
Log In Sign Up

DenseHMM: Learning Hidden Markov Models by Learning Dense Representations

12/17/2020
by   Joachim Sicking, et al.
0

We propose DenseHMM - a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. Our approach enables constraint-free and gradient-based optimization. We propose two optimization schemes that make use of this: a modification of the Baum-Welch algorithm and a direct co-occurrence optimization. The latter one is highly scalable and comes empirically without loss of performance compared to standard HMMs. We show that the non-linearity of the kernelization is crucial for the expressiveness of the representations. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.

READ FULL TEXT
04/11/2012

A Simple Explanation of A Spectral Algorithm for Learning Hidden Markov Models

A simple linear algebraic explanation of the algorithm in "A Spectral Al...
10/24/2017

Learning Hidden Quantum Markov Models

Hidden Quantum Markov Models (HQMMs) can be thought of as quantum probab...
02/19/2018

Learning Hidden Markov Models from Pairwise Co-occurrences with Applications to Topic Modeling

We present a new algorithm for identifying the transition and emission p...
12/20/2019

An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov models

Non-homogeneous hidden Markov models (NHHMM) are a subclass of dependent...
10/31/2018

Clustering-Enhanced Stochastic Gradient MCMC for Hidden Markov Models with Rare States

MCMC algorithms for hidden Markov models, which often rely on the forwar...
11/09/2020

Scaling Hidden Markov Language Models

The hidden Markov model (HMM) is a fundamental tool for sequence modelin...
03/30/2022

A quantum learning approach based on Hidden Markov Models for failure scenarios generation

Finding the failure scenarios of a system is a very complex problem in t...