Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism

by   Peng Su, et al.

With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this paper, we will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. To the best of our knowledge, we are the first to explore this method. The experimental results illustrate that our method improves the BERT model performance and outperforms the state-of-the-art methods on three benchmark datasets for different relation extraction tasks. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model.



There are no comments yet.


page 4


An Empirical Study on Relation Extraction in the Biomedical Domain

Relation extraction is a fundamental problem in natural language process...

Experiments on transfer learning architectures for biomedical relation extraction

Relation extraction (RE) consists in identifying and structuring automat...

Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction

Contrastive learning has been used to learn a high-quality representatio...

Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again

The strong few-shot in-context learning capability of large pre-trained ...

On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning

Deep learning models have achieved state-of-the-art performances on many...

Transformers and the representation of biomedical background knowledge

BioBERT and BioMegatron are Transformers models adapted for the biomedic...

BERT-GT: Cross-sentence n-ary relation extraction with BERT and Graph Transformer

A biomedical relation statement is commonly expressed in multiple senten...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.