Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism

11/01/2020
by   Peng Su, et al.
0

With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this paper, we will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. To the best of our knowledge, we are the first to explore this method. The experimental results illustrate that our method improves the BERT model performance and outperforms the state-of-the-art methods on three benchmark datasets for different relation extraction tasks. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

12/11/2021

An Empirical Study on Relation Extraction in the Biomedical Domain

Relation extraction is a fundamental problem in natural language process...
11/24/2020

Experiments on transfer learning architectures for biomedical relation extraction

Relation extraction (RE) consists in identifying and structuring automat...
04/28/2021

Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction

Contrastive learning has been used to learn a high-quality representatio...
03/16/2022

Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again

The strong few-shot in-context learning capability of large pre-trained ...
11/04/2019

On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning

Deep learning models have achieved state-of-the-art performances on many...
02/04/2022

Transformers and the representation of biomedical background knowledge

BioBERT and BioMegatron are Transformers models adapted for the biomedic...
01/11/2021

BERT-GT: Cross-sentence n-ary relation extraction with BERT and Graph Transformer

A biomedical relation statement is commonly expressed in multiple senten...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.