SINA-BERT: A pre-trained Language Model for Analysis of Medical Texts in Persian

by   Nasrin Taghizadeh, et al.

We have released Sina-BERT, a language model pre-trained on BERT (Devlin et al., 2018) to address the lack of a high-quality Persian language model in the medical domain. SINA-BERT utilizes pre-training on a large-scale corpus of medical contents including formal and informal texts collected from a variety of online resources in order to improve the performance on health-care related tasks. We employ SINA-BERT to complete following representative tasks: categorization of medical questions, medical sentiment analysis, and medical question retrieval. For each task, we have developed Persian annotated data sets for training and evaluation and learnt a representation for the data of each task especially complex and long medical questions. With the same architecture being used across tasks, SINA-BERT outperforms BERT-based models that were previously made available in the Persian language.



There are no comments yet.


page 1

page 2

page 3

page 4


RuBioRoBERTa: a pre-trained biomedical language model for Russian language biomedical text mining

This paper presents several BERT-based models for Russian language biome...

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

We show that BERT (Devlin et al., 2018) is a Markov random field languag...

BERT-Assisted Semantic Annotation Correction for Emotion-Related Questions

Annotated data have traditionally been used to provide the input for tra...

HateBERT: Retraining BERT for Abusive Language Detection in English

In this paper, we introduce HateBERT, a re-trained BERT model for abusiv...

BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA

The BERT language model (LM) (Devlin et al., 2019) is surprisingly good ...

Latent Universal Task-Specific BERT

This paper describes a language representation model which combines the ...

Story Ending Prediction by Transferable BERT

Recent advances, such as GPT and BERT, have shown success in incorporati...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.