Vocabulary Transfer for Medical Texts

08/04/2022
by   Vladislav D. Mosin, et al.
0

Vocabulary transfer is a transfer learning subtask in which language models fine-tune with the corpus-specific tokenization instead of the default one, which is being used during pretraining. This usually improves the resulting performance of the model, and in the paper, we demonstrate that vocabulary transfer is especially beneficial for medical text processing. Using three different medical natural language processing datasets, we show vocabulary transfer to provide up to ten extra percentage points for the downstream classifier accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/29/2021

Fine-Tuning Transformers: Vocabulary Transfer

Transformers are responsible for the vast majority of recent advances in...
research
01/17/2022

MuLVE, A Multi-Language Vocabulary Evaluation Data Set

Vocabulary learning is vital to foreign language learning. Correct and a...
research
03/12/2021

Comparing the Performance of NLP Toolkits and Evaluation measures in Legal Tech

Recent developments in Natural Language Processing have led to the intro...
research
10/26/2021

AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain

During the fine-tuning phase of transfer learning, the pretrained vocabu...
research
11/24/2022

Using Natural Language Processing to Predict Costume Core Vocabulary of Historical Artifacts

Historic dress artifacts are a valuable source for human studies. In par...
research
05/31/2016

Determining the Characteristic Vocabulary for a Specialized Dictionary using Word2vec and a Directed Crawler

Specialized dictionaries are used to understand concepts in specific dom...
research
10/30/2022

Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning

Recent research shows synthetic data as a source of supervision helps pr...

Please sign up or login with your details

Forgot password? Click here to reset