Hierarchically-Refined Label Attention Network for Sequence Labeling

08/23/2019
by   Leyang Cui, et al.
0

CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.

READ FULL TEXT
research
06/20/2019

Few-Shot Sequence Labeling with Label Dependency Transfer

Few-shot sequence labeling faces a unique challenge compared with the ot...
research
07/26/2019

Investigating Self-Attention Network for Chinese Word Segmentation

Neural network has become the dominant method for Chinese word segmentat...
research
12/19/2020

Uncertainty-Aware Label Refinement for Sequence Labeling

Conditional random fields (CRF) for label decoding has become ubiquitous...
research
08/02/2017

Low-Rank Hidden State Embeddings for Viterbi Sequence Labeling

In textual information extraction and other sequence labeling tasks it i...
research
11/11/2020

An Investigation of Potential Function Designs for Neural CRF

The neural linear-chain CRF model is one of the most widely-used approac...
research
11/09/2019

Factored Latent-Dynamic Conditional Random Fields for Single and Multi-label Sequence Modeling

Conditional Random Fields (CRF) are frequently applied for labeling and ...
research
06/12/2018

Design Challenges and Misconceptions in Neural Sequence Labeling

We investigate the design challenges of constructing effective and effic...

Please sign up or login with your details

Forgot password? Click here to reset