DeepAI AI Chat
Log In Sign Up

EmailSum: Abstractive Email Thread Summarization

07/30/2021
by   Shiyue Zhang, et al.
Facebook
Microsoft
University of North Carolina at Chapel Hill
7

Recent years have brought about an interest in the challenging task of summarizing conversation threads (meetings, online discussions, etc.). Such summaries help analysis of the long text to quickly catch up with the decisions made and thus improve our work or communication efficiency. To spur research in thread summarization, we have developed an abstractive Email Thread Summarization (EmailSum) dataset, which contains human-annotated short (<30 words) and long (<100 words) summaries of 2549 email threads (each containing 3 to 10 emails) over a wide variety of topics. We perform a comprehensive empirical study to explore different summarization techniques (including extractive and abstractive methods, single-document and hierarchical models, as well as transfer and semisupervised learning) and conduct human evaluations on both short and long summary generation tasks. Our results reveal the key challenges of current abstractive summarization models in this task, such as understanding the sender's intent and identifying the roles of sender and receiver. Furthermore, we find that widely used automatic evaluation metrics (ROUGE, BERTScore) are weakly correlated with human judgments on this email thread summarization task. Hence, we emphasize the importance of human evaluation and the development of better metrics by the community. Our code and summary data have been made available at: https://github.com/ZhangShiyue/EmailSum

READ FULL TEXT

page 1

page 2

page 3

page 4

10/30/2022

How Far are We from Robust Long Abstractive Summarization?

Abstractive summarization has made tremendous progress in recent years. ...
05/24/2023

SummIt: Iterative Text Summarization via ChatGPT

Existing text summarization systems have made significant progress in re...
08/29/2021

SummerTime: Text Summarization Toolkit for Non-experts

Recent advances in summarization provide models that can generate summar...
09/08/2022

Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization

The problems of unfaithful summaries have been widely discussed under th...
12/17/2022

RISE: Leveraging Retrieval Techniques for Summarization Evaluation

Evaluating automatically-generated text summaries is a challenging task....
03/22/2021

Nutri-bullets: Summarizing Health Studies by Composing Segments

We introduce Nutri-bullets, a multi-document summarization task for heal...