Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation

05/25/2023
by   Hao Li, et al.
6

Argument summarisation is a promising but currently under-explored field. Recent work has aimed to provide textual summaries in the form of concise and salient short texts, i.e., key points (KPs), in a task known as Key Point Analysis (KPA). One of the main challenges in KPA is finding high-quality key point candidates from dozens of arguments even in a small corpus. Furthermore, evaluating key points is crucial in ensuring that the automatically generated summaries are useful. Although automatic methods for evaluating summarisation have considerably advanced over the years, they mainly focus on sentence-level comparison, making it difficult to measure the quality of a summary (a set of KPs) as a whole. Aggravating this problem is the fact that human evaluation is costly and unreproducible. To address the above issues, we propose a two-step abstractive summarisation framework based on neural topic modelling with an iterative clustering procedure, to generate key points which are aligned with how humans identify key points. Our experiments show that our framework advances the state of the art in KPA, with performance improvement of up to 14 (absolute) percentage points, in terms of both ROUGE and our own proposed evaluation metrics. Furthermore, we evaluate the generated summaries using a novel set-based evaluation toolkit. Our quantitative analysis demonstrates the effectiveness of our proposed evaluation metrics in assessing the quality of generated KPs. Human evaluation further demonstrates the advantages of our approach and validates that our proposed evaluation metric is more consistent with human judgment than ROUGE scores.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/11/2020

Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis

When summarizing a collection of views, arguments or opinions on some to...
research
05/04/2020

From Arguments to Key Points: Towards Automatic Argument Summarization

Generating a concise summary from a large collection of arguments on a g...
research
05/13/2021

Towards Human-Free Automatic Quality Evaluation of German Summarization

Evaluating large summarization corpora using humans has proven to be exp...
research
04/21/2022

Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics

How reliably an automatic summarization evaluation metric replicates hum...
research
11/28/2022

Arguments to Key Points Mapping with Prompt-based Learning

Handling and digesting a huge amount of information in an efficient mann...
research
12/23/2022

From Judgement's Premises Towards Key Points

Key Point Analysis(KPA) is a relatively new task in NLP that combines su...
research
07/25/2023

An End-to-End Workflow using Topic Segmentation and Text Summarisation Methods for Improved Podcast Comprehension

The consumption of podcast media has been increasing rapidly. Due to the...

Please sign up or login with your details

Forgot password? Click here to reset