Multimodal Graph-based Transformer Framework for Biomedical Relation Extraction

07/01/2021
by   Sriram Pingali, et al.
0

The recent advancement of pre-trained Transformer models has propelled the development of effective text mining models across various biomedical tasks. However, these models are primarily learned on the textual data and often lack the domain knowledge of the entities to capture the context beyond the sentence. In this study, we introduced a novel framework that enables the model to learn multi-omnics biological information about entities (proteins) with the help of additional multi-modal cues like molecular structure. Towards this, rather developing modality-specific architectures, we devise a generalized and optimized graph based multi-modal learning mechanism that utilizes the GraphBERT model to encode the textual and molecular structure information and exploit the underlying features of various modalities to enable end-to-end learning. We evaluated our proposed method on ProteinProtein Interaction task from the biomedical corpus, where our proposed generalized approach is observed to be benefited by the additional domain-specific modality.

READ FULL TEXT
research
08/14/2023

GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text

Large language models have made significant strides in natural language ...
research
10/19/2022

BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining

Pre-trained language models have attracted increasing attention in the b...
research
12/21/2020

Towards Incorporating Entity-specific Knowledge Graph Information in Predicting Drug-Drug Interactions

Off-the-shelf biomedical embeddings obtained from the recently released ...
research
11/25/2022

Molecular Joint Representation Learning via Multi-modal Information

In recent years, artificial intelligence has played an important role on...
research
05/17/2022

MATrIX – Modality-Aware Transformer for Information eXtraction

We present MATrIX - a Modality-Aware Transformer for Information eXtract...
research
05/25/2023

Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis

Multimodal relation extraction (MRE) is the task of identifying the sema...
research
02/04/2022

Transformers and the representation of biomedical background knowledge

BioBERT and BioMegatron are Transformers models adapted for the biomedic...

Please sign up or login with your details

Forgot password? Click here to reset