Self-attention-based BiGRU and capsule network for named entity recognition

01/30/2020
by   Jianfeng Deng, et al.
0

Named entity recognition(NER) is one of the tasks of natural language processing(NLP). In view of the problem that the traditional character representation ability is weak and the neural network method is unable to capture the important sequence information. An self-attention-based bidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER is proposed. This model generates character vectors through bidirectional encoder representation of transformers(BERT) pre-trained model. BiGRU is used to capture sequence context features, and self-attention mechanism is proposed to give different focus on the information captured by hidden layer of BiGRU. Finally, we propose to use CapsNet for entity recognition. We evaluated the recognition performance of the model on two datasets. Experimental results show that the model has better performance without relying on external dictionary information.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/03/2019

CAN-NER: Convolutional Attention Network forChinese Named Entity Recognition

Named entity recognition (NER) in Chinese is essential but difficult bec...
research
04/10/2021

Not All Attention Is All You Need

Self-attention based models have achieved remarkable success in natural ...
research
08/24/2019

Enhancing Neural Sequence Labeling with Position-Aware Self-Attention

Sequence labeling is a fundamental task in natural language processing a...
research
04/27/2021

UoT-UWF-PartAI at SemEval-2021 Task 5: Self Attention Based Bi-GRU with Multi-Embedding Representation for Toxicity Highlighter

Toxic Spans Detection(TSD) task is defined as highlighting spans that ma...
research
02/18/2022

DataMUX: Data Multiplexing for Neural Networks

In this paper, we introduce data multiplexing (DataMUX), a technique tha...
research
04/08/2020

Self-Attention Gazetteer Embeddings for Named-Entity Recognition

Recent attempts to ingest external knowledge into neural models for name...
research
05/04/2023

The Role of Global and Local Context in Named Entity Recognition

Pre-trained transformer-based models have recently shown great performan...

Please sign up or login with your details

Forgot password? Click here to reset