DeepAI AI Chat
Log In Sign Up

Modeling Topical Coherence in Discourse without Supervision

by   Disha Shrivastava, et al.

Coherence of text is an important attribute to be measured for both manually and automatically generated discourse; but well-defined quantitative metrics for it are still elusive. In this paper, we present a metric for scoring topical coherence of an input paragraph on a real-valued scale by analyzing its underlying topical structure. We first extract all possible topics that the sentences of a paragraph of text are related to. Coherence of this text is then measured by computing: (a) the degree of uncertainty of the topics with respect to the paragraph, and (b) the relatedness between these topics. All components of our modular framework rely only on unlabeled data and WordNet, thus making it completely unsupervised, which is an important feature for general-purpose usage of any metric. Experiments are conducted on two datasets - a publicly available dataset for essay grading (representing human discourse), and a synthetic dataset constructed by mixing content from multiple paragraphs covering diverse topics. Our evaluation shows that the measured coherence scores are positively correlated with the ground truth for both the datasets. Further validation to our coherence scores is provided by conducting human evaluation on the synthetic data, showing a significant agreement of 79.3


Analyzing Neural Discourse Coherence Models

In this work, we systematically investigate how well current models of c...

Towards Modelling Coherence in Spoken Discourse

While there has been significant progress towards modelling coherence in...

RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators

In this paper, we study the task of improving the cohesion and coherence...

W-RST: Towards a Weighted RST-style Discourse Framework

Aiming for a better integration of data-driven and linguistically-inspir...

Subjective Assessments of Legibility in Ancient Manuscript Images – The SALAMI Dataset

The research field concerned with the digital restoration of degraded wr...

On the Contribution of Discourse Structure on Text Complexity Assessment

This paper investigates the influence of discourse features on text comp...

SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours

Media is full of false claims. Even Oxford Dictionaries named "post-trut...