Segmental Recurrent Neural Networks

11/18/2015
by   Lingpeng Kong, et al.
0

We introduce segmental recurrent neural networks (SRNNs) which define, given an input sequence, a joint probability distribution over segmentations of the input and labelings of the segments. Representations of the input segments (i.e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these "segment embeddings" are used to define compatibility scores with output labels. These local compatibility scores are integrated using a global semi-Markov conditional random field. Both fully supervised training -- in which segment boundaries and labels are observed -- as well as partially supervised training -- in which segment boundaries are latent -- are straightforward. Experiments on handwriting recognition and joint Chinese word segmentation/POS tagging show that, compared to models that do not explicitly represent segments such as BIO tagging schemes and connectionist temporal classification (CTC), SRNNs obtain substantially higher accuracies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/15/2021

Neural Sequence Segmentation as Determining the Leftmost Segments

Prior methods to text segmentation are mostly at token level. Despite th...
research
08/13/2018

Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging

Character-level models of tokens have been shown to be effective at deal...
research
05/10/2018

Hybrid semi-Markov CRF for Neural Sequence Labeling

This paper proposes hybrid semi-Markov conditional random fields (SCRFs)...
research
02/24/2017

Sequence Modeling via Segmentations

Segmental structure is a common pattern in many types of sequences such ...
research
02/21/2017

Multitask Learning with CTC and Segmental CRF for Speech Recognition

Segmental conditional random fields (SCRFs) and connectionist temporal c...
research
06/16/2021

Detection of Consonant Errors in Disordered Speech Based on Consonant-vowel Segment Embedding

Speech sound disorder (SSD) refers to a type of developmental disorder i...
research
01/26/2021

Medical Segment Coloring of Clinical Notes

This paper proposes a deep learning-based method to identify the segment...

Please sign up or login with your details

Forgot password? Click here to reset