Log In Sign Up

An LSTM-based Plagiarism Detection via Attention Mechanism and a Population-based Approach for Pre-Training Parameters with imbalanced Classes

by   Seyed Vahid Moravvej, et al.

Plagiarism is one of the leading problems in academic and industrial environments, which its goal is to find the similar items in a typical document or source code. This paper proposes an architecture based on a Long Short-Term Memory (LSTM) and attention mechanism called LSTM-AM-ABC boosted by a population-based approach for parameter initialization. Gradient-based optimization algorithms such as back-propagation (BP) are widely used in the literature for learning process in LSTM, attention mechanism, and feed-forward neural network, while they suffer from some problems such as getting stuck in local optima. To tackle this problem, population-based metaheuristic (PBMH) algorithms can be used. To this end, this paper employs a PBMH algorithm, artificial bee colony (ABC), to moderate the problem. Our proposed algorithm can find the initial values for model learning in all LSTM, attention mechanism, and feed-forward neural network, simultaneously. In other words, ABC algorithm finds a promising point for starting BP algorithm. For evaluation, we compare our proposed algorithm with both conventional and population-based methods. The results clearly show that the proposed method can provide competitive performance.


page 1

page 2

page 3

page 4


Convolutional LSTM Networks for Subcellular Localization of Proteins

Machine learning is widely used to analyze biological sequence data. Non...

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

We propose a simplified model of attention which is applicable to feed-f...

Attention-Based LSTM for Psychological Stress Detection from Spoken Language Using Distant Supervision

We propose a Long Short-Term Memory (LSTM) with attention mechanism to c...

FRA-LSTM: A Vessel Trajectory Prediction Method Based on Fusion of the Forward and Reverse Sub-Network

In order to improve the vessel's capacity and ensure maritime traffic sa...

Multi-layer Attention Mechanism for Speech Keyword Recognition

As an important part of speech recognition technology, automatic speech ...

End-to-End Speaker Height and age estimation using Attention Mechanism with LSTM-RNN

Automatic height and age estimation of speakers using acoustic features ...

p-DLA: A Predictive System Model for Onshore Oil and Gas Pipeline Dataset Classification and Monitoring - Part 1

With the rise in militant activity and rogue behaviour in oil and gas re...