DeepAI AI Chat
Log In Sign Up

Raw-to-End Name Entity Recognition in Social Media

by   Liyuan Liu, et al.

Taking word sequences as the input, typical named entity recognition (NER) models neglect errors from pre-processing (e.g., tokenization). However, these errors can influence the model performance greatly, especially for noisy texts like tweets. Here, we introduce Neural-Char-CRF, a raw-to-end framework that is more robust to pre-processing errors. It takes raw character sequences as inputs and makes end-to-end predictions. Word embedding and contextualized representation models are further tailored to capture textual signals for each character instead of each word. Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries. Moreover, we observe our model performance remains unchanged after replacing tokenization with string matching, which demonstrates its potential to be tokenization-free. Extensive experimental results on two public datasets demonstrate the superiority of our proposed method over the state of the art. The implementations and datasets are made available at:


page 1

page 2

page 3

page 4


A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition

We propose a morphologically informed model for named entity recognition...

MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition

Recently, word enhancement has become very popular for Chinese Named Ent...

NFLAT: Non-Flat-Lattice Transformer for Chinese Named Entity Recognition

Recently, Flat-LAttice Transformer (FLAT) has achieved great success in ...

A Multi-task Approach for Named Entity Recognition in Social Media Data

Named Entity Recognition for social media data is challenging because of...