Improving NER's Performance with Massive financial corpus

07/31/2020
by   Han Zhang, et al.
0

Training large deep neural networks needs massive high quality annotation data, but the time and labor costs are too expensive for small business. We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with minimum labor cost. The methods we use involve pre-training a lite language model such as Albert-small or Electra-small in financial corpus, knowledge of distillation and multi-stage learning. The result is that we raised the recall rate by nearly 20 points and get 4 times as fast as BERT-CRF model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/01/2021

NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging

Named entity recognition (NER) models generally perform poorly when larg...
research
03/03/2020

CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model

In this paper, we introduce the Chinese corpus from CLUE organization, C...
research
08/29/2023

SpikeBERT: A Language Spikformer Trained with Two-Stage Knowledge Distillation from BERT

Spiking neural networks (SNNs) offer a promising avenue to implement dee...
research
04/12/2020

TinyMBERT: Multi-Stage Distillation Framework for Massive Multi-lingual NER

Deep and large pre-trained language models are the state-of-the-art for ...
research
04/23/2021

Transfer training from smaller language model

Large language models have led to state-of-the-art accuracies across a r...
research
04/03/2023

Efficient human-in-loop deep learning model training with iterative refinement and statistical result validation

Annotation and labeling of images are some of the biggest challenges in ...
research
10/11/2017

Decision support from financial disclosures with deep neural networks and transfer learning

Company disclosures greatly aid in the process of financial decision-mak...

Please sign up or login with your details

Forgot password? Click here to reset