A Semantic QA-Based Approach for Text Summarization Evaluation

by   Ping Chen, et al.

Many Natural Language Processing and Computational Linguistics applications involves the generation of new texts based on some existing texts, such as summarization, text simplification and machine translation. However, there has been a serious problem haunting these applications for decades, that is, how to automatically and accurately assess quality of these applications. In this paper, we will present some preliminary results on one especially useful and challenging problem in NLP system evaluation: how to pinpoint content differences of two text passages (especially for large pas-sages such as articles and books). Our idea is intuitive and very different from existing approaches. We treat one text passage as a small knowledge base, and ask it a large number of questions to exhaustively identify all content points in it. By comparing the correctly answered questions from two text passages, we will be able to compare their content precisely. The experiment using 2007 DUC summarization corpus clearly shows promising results.



There are no comments yet.


page 4


Un résumeur à base de graphes, indépéndant de la langue

In this paper we present REG, a graph-based approach for study a fundame...

LCSTS: A Large Scale Chinese Short Text Summarization Dataset

Automatic text summarization is widely regarded as the highly difficult ...

VieSum: How Robust Are Transformer-based Models on Vietnamese Summarization?

Text summarization is a challenging task within natural language process...

BARTScore: Evaluating Generated Text as Text Generation

A wide variety of NLP applications, such as machine translation, summari...

Automatic Text Summarization of Legal Cases: A Hybrid Approach

Manual Summarization of large bodies of text involves a lot of human eff...

A Preliminary Study for Building an Arabic Corpus of Pair Questions-Texts from the Web: AQA-Webcorp

With the development of electronic media and the heterogeneity of Arabic...

Text similarity analysis for evaluation of descriptive answers

Keeping in mind the necessity of intelligent system in educational secto...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.