Introducing the Hidden Neural Markov Chain framework

02/17/2021
by   Elie Azeraf, et al.
0

Nowadays, neural network models achieve state-of-the-art results in many areas as computer vision or speech processing. For sequential data, especially for Natural Language Processing (NLP) tasks, Recurrent Neural Networks (RNNs) and their extensions, the Long Short Term Memory (LSTM) network and the Gated Recurrent Unit (GRU), are among the most used models, having a "term-to-term" sequence processing. However, if many works create extensions and improvements of the RNN, few have focused on developing other ways for sequential data processing with neural networks in a "term-to-term" way. This paper proposes the original Hidden Neural Markov Chain (HNMC) framework, a new family of sequential neural models. They are not based on the RNN but on the Hidden Markov Model (HMM), a probabilistic graphical model. This neural extension is possible thanks to the recent Entropic Forward-Backward algorithm for HMM restoration. We propose three different models: the classic HNMC, the HNMC2, and the HNMC-CN. After describing our models' whole construction, we compare them with classic RNN and Bidirectional RNN (BiRNN) models for some sequence labeling tasks: Chunking, Part-Of-Speech Tagging, and Named Entity Recognition. For every experiment, whatever the architecture or the embedding method used, one of our proposed models has the best results. It shows this new neural sequential framework's potential, which can open the way to new models, and might eventually compete with the prevalent BiLSTM and BiGRU.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/21/2020

Hidden Markov Chains, Entropic Forward-Backward, and Part-Of-Speech Tagging

The ability to take into account the characteristics - also called featu...
research
05/18/2018

Suffix Bidirectional Long Short-Term Memory

Recurrent neural networks have become ubiquitous in computing representa...
research
02/07/2021

Few-shot time series segmentation using prototype-defined infinite hidden Markov models

We propose a robust framework for interpretable, few-shot analysis of no...
research
02/17/2021

Highly Fast Text Segmentation With Pairwise Markov Chains

Natural Language Processing (NLP) models' current trend consists of usin...
research
04/05/2019

Diversified Hidden Markov Models for Sequential Labeling

Labeling of sequential data is a prevalent meta-problem for a wide range...
research
01/11/2017

Decoding with Finite-State Transducers on GPUs

Weighted finite automata and transducers (including hidden Markov models...
research
12/11/2017

A Novel Way of Identifying Cyber Predators

Recurrent Neural Networks with Long Short-Term Memory cell (LSTM-RNN) ha...

Please sign up or login with your details

Forgot password? Click here to reset