λ-Scaled-Attention: A Novel Fast Attention Mechanism for Efficient Modeling of Protein Sequences

01/09/2022
by   Ashish Ranjan, et al.
0

Attention-based deep networks have been successfully applied on textual data in the field of NLP. However, their application on protein sequences poses additional challenges due to the weak semantics of the protein words, unlike the plain text words. These unexplored challenges faced by the standard attention technique include (i) vanishing attention score problem and (ii) high variations in the attention distribution. In this regard, we introduce a novel λ-scaled attention technique for fast and efficient modeling of the protein sequences that addresses both the above problems. This is used to develop the λ-scaled attention network and is evaluated for the task of protein function prediction implemented at the protein sub-sequence level. Experiments on the datasets for biological process (BP) and molecular function (MF) showed significant improvements in the F1 score values for the proposed λ-scaled attention technique over its counterpart approach based on the standard attention technique (+2.01 state-of-the-art ProtVecGen-Plus approach (+2.61 Further, fast convergence (converging in half the number of epochs) and efficient learning (in terms of very low difference between the training and validation losses) were also observed during the training process.

READ FULL TEXT
research
06/27/2022

ProGen2: Exploring the Boundaries of Protein Language Models

Attention-based models trained on protein sequences have demonstrated in...
research
11/04/2018

Deep Robust Framework for Protein Function Prediction using Variable-Length Protein Sequences

Amino acid sequence portrays most intrinsic form of a protein and expres...
research
12/06/2017

Attention based convolutional neural network for predicting RNA-protein binding sites

RNA-binding proteins (RBPs) play crucial roles in many biological proces...
research
01/07/2020

Knowledge-aware Attention Network for Protein-Protein Interaction Extraction

Protein-protein interaction (PPI) extraction from published scientific l...
research
07/05/2018

Feature Assisted bi-directional LSTM Model for Protein-Protein Interaction Identification from Biomedical Texts

Knowledge about protein-protein interactions is essential in understandi...
research
03/14/2022

A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification

Many recent deep learning-based solutions have widely adopted the attent...
research
06/25/2017

Finding optimal finite biological sequences over finite alphabets: the OptiFin toolbox

In this paper, we present a toolbox for a specific optimization problem ...

Please sign up or login with your details

Forgot password? Click here to reset