DeepAI AI Chat
Log In Sign Up

Experiments on transfer learning architectures for biomedical relation extraction

by   Walid Hafiane, et al.

Relation extraction (RE) consists in identifying and structuring automatically relations of interest from texts. Recently, BERT improved the top performances for several NLP tasks, including RE. However, the best way to use BERT, within a machine learning architecture, and within a transfer learning strategy is still an open question since it is highly dependent on each specific task and domain. Here, we explore various BERT-based architectures and transfer learning strategies (i.e., frozen or fine-tuned) for the task of biomedical RE on two corpora. Among tested architectures and strategies, our *BERT-segMCNN with finetuning reaches performances higher than the state-of-the-art on the two corpora (1.73 ChemProt and PGxCorpus corpora respectively). More generally, our experiments illustrate the expected interest of fine-tuning with BERT, but also the unexplored advantage of using structural information (with sentence segmentation), in addition to the context classically leveraged by BERT.


page 1

page 2

page 3

page 4


Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism

With the explosive growth of biomedical literature, designing automatic ...

Transfer Learning for Causal Sentence Detection

We consider the task of detecting sentences that express causality, as a...

Does constituency analysis enhance domain-specific pre-trained BERT models for relation extraction?

Recently many studies have been conducted on the topic of relation extra...

An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale

This paper conducts an empirical investigation to evaluate transfer lear...

Detecting Insincere Questions from Text: A Transfer Learning Approach

The internet today has become an unrivalled source of information where ...

On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning

Deep learning models have achieved state-of-the-art performances on many...