DeepAI AI Chat
Log In Sign Up

A Deep Network Model for Paraphrase Detection in Short Text Messages

12/07/2017
by   Basant Agarwal, et al.
NTNU
0

This paper is concerned with paraphrase detection. The ability to detect similar sentences written in natural language is crucial for several applications, such as text mining, text summarization, plagiarism detection, authorship authentication and question answering. Given two sentences, the objective is to detect whether they are semantically identical. An important insight from this work is that existing paraphrase systems perform well when applied on clean texts, but they do not necessarily deliver good performance against noisy texts. Challenges with paraphrase detection on user generated short texts, such as Twitter, include language irregularity and noise. To cope with these challenges, we propose a novel deep neural network-based approach that relies on coarse-grained sentence modeling using a convolutional neural network and a long short-term memory model, combined with a specific fine-grained word-level similarity matching model. Our experimental results show that the proposed approach outperforms existing state-of-the-art approaches on user-generated noisy social media data, such as Twitter texts, and achieves highly competitive performance on a cleaner corpus.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/09/2015

Syntax-based Deep Matching of Short Texts

Many tasks in natural language processing, ranging from machine translat...
08/30/2016

Language Detection For Short Text Messages In Social Media

With the constant growth of the World Wide Web and the number of documen...
11/09/2020

Automated Discovery of Mathematical Definitions in Text with Deep Neural Networks

Automatic definition extraction from texts is an important task that has...
04/26/2020

Detect Language of Transliterated Texts

Informal transliteration from other languages to English is prevalent in...
07/23/2019

Happiness Entailment: Automating Suggestions for Well-Being

Understanding what makes people happy is a central topic in psychology. ...
12/29/2017

Methods for Detecting Paraphrase Plagiarism

Paraphrase plagiarism is one of the difficult challenges facing plagiari...
05/26/2022

Leveraging Dependency Grammar for Fine-Grained Offensive Language Detection using Graph Convolutional Networks

The last few years have witnessed an exponential rise in the propagation...